Intelligence gathered by members of the multi-agency Project Bloom group, which was set up to tackle pension scams, has found some people who had put away more than £1m had lost their retirement funds to criminals.
New Action Fraud data revealed that two people reported that they had lost the seven-figure sums.
But the The Pensions Regulator (TPR) believes this could be the tip of the iceberg it is believed the majority of scam victims never contact the authorities.
On average, victims of pension scams lost £91,000 each to fraudsters in 2017.
They reported receiving cold-calls, offers of free pension reviews and promises that they would get high rates of return - all of which are key warning signs of scams, TPR warned.
A ban on pension cold calling came into force earlier this month and firms which break the rules could face penalties of up to £500,000.
Research conducted by the FCA highlighted that “a large proportion of pension holders consider themselves to be too savvy to be scammed.”
Nicola Parish, TPR’s executive director of frontline regulation, said: “Victims of scams are often traumatised by what has happened to them and many inevitably are left questioning how they are going to afford to retire.
“The average loss of a victim is £91,000 but these Action Fraud reports show that people can also lose much, much more.
“However large your pension pot, you must be vigilant and able to spot and avoid a scam.”
Pauline Smith, director of Action Fraud, said: “These statistics prove that the consequences of falling victim to a pension scam can be devastating.
“Victims can lose their life savings and are left facing retirement with little or no income.
“This is why it’s so important that you are vigilant if you receive an offer about your pension out of the blue and that you check who you are dealing with.
“If you think you have been a victim of pension fraud, please report it to us.”
The partners in Bloom will meet this week to discuss the scams problem and how they can work closer together to smash the scammers.