The Grade I listed 14th-century hidden gem castle is next to a riverside and has some remarkably well-preserved elements.