The owners of a cafe that has been serving a town for more than three decades say “the time has come” for them to say goodbye to the business.