One Birmingham councillor said he was worried about the 'incremental reduction in the character of the Jewellery Quarter'