Sussan Ley has cast doubt on a key claim made by Anthony Albanese in justifying his rejection of a federal inquiry into anti-Semitism.