A report found the presence of the Cardiff-raised peer on the Parliamentary estate 'would present a clear and intolerable risk'