US Navy Secretary John Phelan is leaving the Trump administration, the Pentagon announced on Wednesday.