My personal world view is that it is because housing has become financialized. In this situation the rent probably doesn't even cover the fixed ownership costs for the property (caveat: prop 13).

So why are investors clamoring over houses that have a negative rate of return? The speculation that the value will increase. We should do everything in our power to make real estate a non-appreciating asset. If that means government owned entities constructing social housing as needed, sign me up.