The little red house was once lived in by a 6ft 3in fisherman, until the council deemed it unfit for habitation