If you want to live in a very desirable area and you also want to have very few neighbors, you should have to pay for that luxury. Right now that luxury is effectively subsidized for a few people who are “grandfathered in”, thanks to the market distortion created by zoning.