The former environment minister entered the House of Commons on Wednesday to offer her support to her former colleague