A proposed high-speed rail line between Toronto and Quebec City is facing growing opposition from some rural Ontarians and Quebecers.