At a cabinet meeting, President Trump claimed that the military wanted the ballroom ‘more than anybody‘