The celebrity chef previously told PEOPLE that he considers Prince Harry and Meghan Markle to be his "friends"