Ed Miliband said he told David Lammy he thought the appointment could 'blow up' - and the then-Foreign Secretary shared his concerns.