The prime minister hits out at those who took issue with Hannah Yeoh's appointment as federal territories minister.