It's super frustrating when people take a phrase, change its meaning by removing it from context, and then nitpick it.

That the choices are ones you don't like doesn't mean they aren't choices. The people who live in Munger all did so voluntarily, meaning that they're going to be the people who were most likely to be ok with Munger's limitations.