Correct. Developers build to what the market wants. But that doesn't mean they build to what people want. If given total freedom, developers often chase the upscale market, rich people wanting high-priced units. That is where profit is to be made. That is what the market wants. But that is not necessarily what a city actually needs.
If you make housing construction unprofitable, then fewer people will build houses.
Also, people act like a glut of luxury housing would be a problem. It absolutely wouldn't. It would displace and therefore depress prices for midrange units. If the developers end up losing money selling their new luxury housing at a loss, then they've effectively just subsidized the US housing supply.
That's much less expensive for taxpayers than having the government subsidize + force people to build housing that sells for less than construction costs.
The one large house + not enough houses problem is due to income inequality. Making houses scarce increases the price of houses for everyone, so it exacerbates income inequality (it is hard to increase your income and educate your kids while living in a tent). Worse, most affordable housing projects that are currently being proposed deed-restrict the housing (otherwise, unless the initial purchaser is an idiot, they'd immediately sell at a large profit). That prevents poor people from profiting off of the appreciation of property in their neighborhood that and therefore increases long-term income inequality. You also get situations where the low income housing owner can't afford to use local amenities, or maintain their house (which they bought for far less than the cost of initial construction).
The second issue (hoarding of vacant houses) can be solved with a vacancy tax.
History supports both those arguments. Can you name an example of a city where supply has exceeded demand (measured in months of housing inventory on the market) for an extended period of time (multiple years, so longer than the 2008 crisis), and that also has a housing affordability and homeless crisis?
Similarly, can you cite an example of a market where housing has been freely built and sold at a profit over the long term, and that also has a housing supply problem?