The proportion of young people who identify with either party is declining, according to new polling.