I dunno why this is being downvoted because in my experience this is spot on, and I've heard such things from my neighbours. (maybe people are uncomfortable with the truth spoken so plainly)

Most of all people want exclusivity. Only having a handful of neighbours is something that is hard to buy. It's easier enforced through regulation.

Established neighbourhoods push for regulations to keep people out.

In the past it absolutely was for racist reasons. Today more likely economic ones. Apartments means renters which means less wealthy people. Exclusive detached home areas are pretty much guaranteed to have a high floor of wealth.