It is understood that a portrait of the late Queen was removed shortly after former Foreign Secretary took office in July 2024.