That also means "those people" can walk into your neighborhood, and many are motivated not to let that happen.

I lived in an area that, for years, refused to let an old railroad track be converted into a trail because it would connect a working class area of the town to an upper middle class area via a miles-long footpath.