Patrick O’Donovan said his relationship with Ross had been “rather difficult” when he was a junior minister in the department.