Ex-No 10 official was amazed the Prime Minister called bringing Egyptian radical to the UK a priority