The 36-year-old music producer - whose real name is Shahid Khan - faced radio DJ Snoochie Shy in the Treacherous Traps Bushtucker Trial, after being picked by the contestants from the Main Camp.
After losing the trial Naughty Boy told his fellow campmates in The Clink: "I do think my time here is done. I'm sorry.
"I had a feeling earlier and I was thinking about mum. Obviously I love you guys, but I don’t think after knowing what the Main Camp probably knew what they were putting me through knowing I did the challenge earlier – I don’t know if I could ever join that camp.”
And 78-year-old former Strictly Come Dancing judge Dame Arlene Phillips then revealed she had also had enough and was ready to throw in the towel, saying: "I feel the same."
Snoochie and Naughty Boy were locked in coffins filled with live rats and cockroaches and had to race to free themselves in order to win meals for their campmates.
The Radio 1 Xtra DJ had been voted to take part in the challenge by the public but the music producer was chosen by his rival campmates.
Naughty Boy insisted he wanted to leave, saying: “This isn’t a random big decision, I've been thinking earlier on today.
"I’ve tried my best both times today. I just can’t go from here into that Main Camp. I wear my heart on my face, I can’t fake that. I’m sorry.
"One thing I’ll tell you before you guys go into the Main Camp, as much as they say they love us, they love you… there is no love. When you guys go to the Main Camp, they’re the reason why I left.”
They were then told they would be joining the Main Camp in the morning and Madeley did his best to persuade Naughty Boy and Phillips to sleep on it.
Naughty Boy said: "I don't think I'll be coming to the Main Camp. I can't continue something that my heart's not in. I cant join a camp that don't give a s*** about us. I think my experience, as amazing as it's been - I think it's over."
Phillips said: "I thought I was going to come out here strong and proud. I just don't think anything will help me stay. I'm happy to sleep on it but I won't change my mind.."
But hosts Ant McPartlin and Declan Donnelly revealed the pair both changed their minds in the morning and tomorrow night's show will see them confront the Main Camp celebrities.
Naughty Boy - who has worked with stars including Beyonce, Sam Smith, Britney Spears and One Direction's Zayn Malik - has previously revealed he signed up to the reality show to help raise awareness for charity Dementia UK, inspired by his 68-year-old mother. She was diagnosed with vascular dementia after suffering a stroke in 2017.
He said: "I live with Mum, and I am used to seeing her all the time. Mum hasn’t been very well, and it is going to be difficult for me being away. But that said, I think the break will be good for both of us.”
Watch: Who Is Naughty Boy?