The people who once called the 'lost village' home were never allowed to return, and now all that survives is a few uninhabited buildings