Locals expressed their fears about potential 'over-development' of the land where the man wanted to build a shed and garden room