Shadow Local Government Secretary Sir James Cleverly said taxpayers "should not be paying full rates for part-time services".