Furthermore, when you buy a house, you are a taking a risk. You're making an investment. I don't see why the rest of the world should be bending over backwards and living in stasis to protect your investment. If you want absolute control over what happens on your block, buy your block.
You're treating all real estate as if it's the same, but the world is more complex than that. We're talking about dense metro areas, where development primarily happens to support growing populations and an influx of businesses. How many expensive homes in downtown SF have become worthless because of industrial ventures nearby? Far, far, far less than have become extremely valuable, if any. This isn't just some theory on paper.
In short there are a lot of reasons to want to regulate what people build.