Trump expressed frustration that his party could lose the House or Senate in the upcoming midterms, the report says