Lord Robert Hayward said the Prime Minister was on course for losses in England, Scotland and Wales at the ballot box