Four Birmingham councillors quit the Labour Party in the past week amid concerns over the direction of the city council