A man alleges he was abused by an authority figure while playing for an Ipswich and West Moreton primary schools squad.