‘We would talk about the difference in fabrics for seasons,’ a former Trump administration official said of the deputy chief of staff