This is the core problem. Commoditization, or rather, people cornering the market is secondary.
City government is doing this because it is elected by those who own single family units. This is self-stabilizing.
If the majority would live in multi residential highrises, there wouldn't be an incentive to artificially create scarcity. Instead, there would be an incentive to drop housing costs even further by increasing supply.
The problem is that nobody wants to invest in a city with a falling housing market. There needs to be some profit sharing between those who build the housing and those who own the real estate in the city center whose value increases when the city grows.
The thing that can really crater a housing market is a departure of jobs from an area.
How do you protect your investment when there is innovative pressure and housing units compete on price? Say you have a building and you try to monetize your investment over 20 years but after ten years, due to innovation, competing units can be built for half your costs. Then your tenants will move if you don't halve your rent which destroys your profits.
In a city with nimby's, that's never a problem because rent remains high, even if costs are reduced.
>All we need to do is force cities to zone land this way
This requires the vote of the nimby's which means that you need other options.
regarding your first point:The loss from rezoning will happen at the time of rezoning and will primarily affect the currently zoned high density land. With newly repriced lower, non scarce multi unit residential land it is unlikely to end up in a large downward risk situation because the sophisticated reits will be pricing what they will pay now for land off of what they think will happen over the next twenty years and will likely be pricing in a rent drop (which is what drives the drop in land prices for high density). You also aren't really at risk of technological disruption because the total turnover/increase in units is low relative to total housing stock even if we ented a building boom. If it suddenly costs half as much to build a high density residential building, it's still going to take decades for that to erode rental costs by half because you don't suddenly have enough supply of that cheaper to build stock to house everyone.
The problem with this is the only entities with the financial power to build such buildings are large developers and corporate landlords. That leads to either high rents or high HOA fees. We need some way to pool prospective homeowners together to build this kind of high-density housing. Because otherwise we’re trading one problem for another and still failing to make home ownership affordable.
> We need some way to pool
That's e.g. a co-op, not exactly unusual concept to pay for housing construction at different scales.
I already explained why prices go up when there is sufficient supply: hyper-fueled demand from essentially free money and new entrants like hedge funds looking for ways to invest the loads of cash they can borrow for nothing and money from their investors as well. In many cities, more than 30% of all home purchases were from investors.
The price rises without new increase in the supply, because you would not lock-in to a particular house on just your first visit. So, each of these visits from each person constitutes each time to total demand. And demand gets a bit inflated. Speculative nature of real estate on top of that, encourages the sellers to hoard with higher prices (like a synthetic monopoly mindset).
Even if one buyer locks into a higher price, the effective market rate goes up because every seller in that area expects much higher price than that.
Price based on Supply and Demand only work properly in a Open Market with High Liquidity.
The market isn't failing to function because someone realized they could buy up current stock and make a large profit over time, people noticed they could make a large profit over time because city governments zoned everything wrong for the actual demand.