Both local governments and popular opinion in those regions differ from your opinion, although you are still entitled to that opinion.

Local residents and voters have the right to keep others out if their laws allow for it. Outsiders/non-citizens don't get a vote.