Labour's controversial 'mansion tax' plan will force thousands to downsize and drive many more into a bureaucratic nightmare of legal appeals, an official study shows.