The administration has been accused of trying to ‘wash their hands of these folks’ with the reported plans