Buy-to-let has ballooned over the last 15 years, but this is now sliding into reverse as landlords fear increased regulation.