Angus Taylor has told Australian permanent residents there “is a price you pay” if they don’t become citizens.