First, I've never heard of anyone being forced to sell, unless there's some truly huge and necessary infrastructure development. You are generally offered some money by a developer and you personally decide whether or not to take it. Sometimes people can be stubborn and there are plenty of examples of large high-rises built around tiny structures that wouldn't sell.
Also, your price per square foot goes up fastest directly adjacent to the largest developments. If your neighborhood can economically support larger buildings, that directly increases the amount of money developers will pay for your land. The less development you allow your neighborhood to have, the less your neighborhood will be able to support large developments. You want your neighborhood to support large developments.
You may want to hold on to YOUR house as an investment, and to keep from a property tax re-assessment, but it makes no economic sense to prevent your neighbors from building higher if they want to.
Maybe not in the USA but certain countries have a special emergency nationalisation procedure that has payment first, skips the consent and leaves the challenge to after-the-fact.
If instead my neighborhood is converted to apartments or high-density condos, I might not have the capital to take advantage of those opportunities but my single family house becomes undesirable in that zone. I am forced to sell to a developer in order to maximize return. I probably still make money, but it might be less then I could have made in the other scenario, and I am unable to use the house for my own residence up until the time I need to liquidate.
So its not necessarily illogical.