The couple were at the centre of several headlines that speculated about the state of their relationship after Spencer announced that he was leaving their podcast ‘Spencer & Vogue’