Collier, 57, a former Detective Chief Inspector with the Metropolitan Police, was banished from the series last week