Prime Minister Anthony Albanese says it's now up to the coalition to justify its position should it oppose amended hate speech legislation.