The celebrity chef first bought his home there back in 2002, and it's loved by a whole host of other famous faces like Keira Knightley and Louis Theroux