Olympic super-heavyweight champion Joshua fights Whyte for the vacant belt at London's O2 Arena on 12 December.
Whyte also called Commonwealth champion Joshua "a scumbag" and "a fake".
"People say 'it was the amateurs, it doesn't matter'. But he was on the end of those punches, so it's in the back of his mind," Whyte told BBC Sport.
"Pumping up your muscles and getting bigger and stronger is one thing, a boxer's mentality is another.
"He's used to being the bully and being on top. But what happens when he tries to impose himself and he keeps being denied? We'll find out soon."
Watford's Joshua, 26, has knocked out all 14 of his professional opponents and none of them has made it past the third round.
In his last fight in September, Joshua won the Commonwealth title courtesy of a 97-second knockout of Scotland's Gary Cornish.
But while Londoner Whyte respects Joshua's abilities in the ring, he is less complimentary about Joshua as a person.
"He pretends to be this good guy but he's got no manners and speaks bad about fighters he's beaten," said Whyte, referring to comments Joshua made about Paul Butlin, a previous victim.
"OK, you won a gold medal at the Olympics, you're going well, people are happy for you. But don't look down on those who have had a harder route than you.
"He preaches to stay humble, but the way he preaches it's like he's trying to convince himself he's a good person."
Whyte, 27, is no stranger to controversy, having been banned for two years in 2012 after testing positive for a banned stimulant.
However, he is undefeated in 16 professional contests, with 13 knockouts, and is now handled by Wladimir Klitschko's trainer Johnathon Banks.
And Whyte believes sharing a training camp with Klitschko, who defends his world titles against Britain's Tyson Fury on 28 November, has greatly improved him as a boxer.
"You pick up a lot of experience being around the best heavyweight of the last 10 years," said Whyte, speaking from Klitschko's camp in the Austrian Alps.
"And Johnathon Banks is a very important part of my game now. It's all about technique, being faster, being more accurate.
"I'm completely different to everyone Anthony Joshua has fought. Dillian Whyte will be victorious. I'll go for the win whichever way I can get it but I definitely want to knock him out."
