Vornado Realty Trust chairman Steven Roth told investors he thinks the phrase is "just as hateful as some disgusting racial slurs."