Richard Osman thought his House of Games was in line for a BAFTA, despite previously admitting it 'could not matter less'