It offers stunning coastal views, a thriving high street and a rich history - and it's perfect even in the rain