This market town saw a surge in visitors thanks to its links to Harry Potter, and in addition to its iconic castle, it has a charming historic town centre