Prime Minister Anthony Albanese may have to go back to the drawing board on contentious hate speech reforms as the Greens say they cannot back the proposal.