It's one of the best daffodil gardens in the UK and visitors can tour a former queen's orchard in 'rare access' opportunity