The pop star, 34, said he losing his job on the show in 2015 due to a high-profile blunder really knocked his confidence.
Olly shot to fame on the TV talent show in 2009, and returned six years later to co-host the series with Caroline Flack.
However, his stint as a presenter only last for one season.
Getting dropped was said to have been linked to the embarrassing moment he accidentally announced a contestant had been booted before it actually happened.
Olly has now revealed the experience left him battling insecurities.
He told the Daily Star: "I have struggled [with mental health problems]…
"The X Factor mistake affected me mentally a lot more than I first realised…
"Over the months afterwards I started to lose confidence during live TV or performances.
“It was like I was no longer on auto-pilot and instead I would have self-doubt and fear I would make a mistake or do something wrong.
“I got through it but it had a knock-on effect on me for a long time.
“There’s lots of mental health problems in this industry. You have to be very mentally tough and thick-skinned.
“I think everyone struggles with mental health at some point."
Olly is taking part in the Great North Run today raising money for mental health charity MIND.
Got a story? email [email protected] or call us direct on 02077824220.
We pay for videos too. Click here to upload yours.
Source: Read Full Article