The housing supply is naturally inelastic, meaning, the ability to increase it is limited. For example, currently a significant part of the US construction capacity is tied up in building data-centers, increasing it is near impossible at anything approaching a reasonable cost, and even if it was possible, doing it would create problems in the future - this is a proven fact, not a conjecture.
> removing market speculation may produce some changes in price, but they're not causing the underlying lack of houses, merely profiting off of if.
You aren't connecting the dots. As it is today, both housing and data-centers are largely driven by speculation - on a mass scale. You severely underestimate the scope and role of speculation in today's markets - yes, we can dream about some imaginary world where competition, demand and supply solve everything but the reality is very different.
Imagine housing in these markets was like cars (or housing in Japan). Houses are worth less every year, no one wants to buy a old house, housing is relatively plentiful + cheap, rents are low.
Private equity wouldn't buy houses even if there was financing since they would lose money. No one would keep their first house and rent it out as they accumulate more houses, because they'd lose money.
In fact if you started doing those activities it wouldn't change the housing market, you'd eventually lose enough money that you would have to stop.
Given a fixed housing supply, increasing demand, and no ability to increase the housing supply, removing market speculation may produce some changes in price, but they're not causing the underlying lack of houses, merely profiting off of if. An effective way to drive them out is to build a lot of housing (aka fix the market).