Police had arrested 12 people and were investigating anti-Israeli chants allegedly made at the rally.