Pro and anti-immigration demonstrators chanted at each other as they were kept apart by barriers and police.