A pensioner accused of sexually assaulting a young girl has died leaving his alleged victim and family devastated that he never faced trial.