The FCA's Tracey McDermott will be taking over as acting chief executive from 12 September while the regulator searches for a permanent replacement.
The FCA says that Mr Wheatley will continue to act as an adviser to the FCA Board until January 2016 with "a particular emphasis on the implementation of the Fair and Effective Markets Review, which he co-chaired."
Mr Wheatley's total remuneration package in the year ended March 2015 amounted to £701,000 including pay, bonuses and benefits.
The FCA has not disclosed Mr Wheatley's reason for leaving but it's believed a change of leadership was seen as necessary.
The BBC quoted Chancellor George Osborne today thanking Mr Wheatley for his service, adding he had done "a brilliant job of launching the FCA in tough circumstances" and adding: "The government believes that different leadership is required to build on those foundations."
John Griffith-Jones, chairman of the FCA said: “Martin has done an outstanding job as chief executive setting up and leading the FCA over the last four years.
"We owe him a lot and I and my board would like to thank him for his great efforts in setting up the organisation and for the contribution he has made to putting conduct so firmly at the top of the financial services agenda. We all wish Martin well and I am pleased that we will continue to benefit from his wisdom and expertise over the next few months.”
Mr Wheatley said: “I am incredibly proud of all we have achieved together in building the FCA over the last four years. I know that the organisation will build on that strong start and work so that the financial services industry continues to thrive.”
Mr Griffith-Jones also announced today that Tracey McDermott will be taking over as acting chief executive from 12 September while the chief executive recruitment process takes place.
He said: “I am pleased that we are able to call on someone of Tracey’s ability and stature to take up the post of acting chief executive. I am grateful to her for taking on this role and I know she will do a great job.”
The FCA became responsible for the conduct supervision of all regulated financial firms and the prudential supervision of those not supervised by the Prudential Regulation Authority (PRA) on 1 April 2013.