Record-breaking hot weather at the end of May contributed to a 2.6% decline in visits to retail destinations, compared to the same month last year.