The high rate of property inflation is pricing thousands of young people out of the home towns and villages they grew up in, an expert said.