With cost of living pains growing by the day, stark new polling shows voters are starting to turn away from the Labor government in droves.