http://urbankchoze.blogspot.com/2014/04/japanese-zoning.html
TL;DR: You can build housing almost anywhere if there exists demand.
I also often wonder if something like a hybrid employee owned business+occupant owned housing could ever be bootstrapped. Something to function like a small village in the modern era/larger surrounding. But that wouldn't just be a hippy/cult commune.
Nope, the finance agreements keep that space empty.
Missing rent can generally be tacked onto the end of your financing. Lower rent causes the basis of your finance agreement to change and you will have to cough up cash.
So, owners are willing to let spaces sit empty for years to avoid having to paying cash (which they probably don't have) to the bank.
It's all musical chairs until higher interest rates cause a few failures which cascades through the entire system.
Good to see SF at least is moving in that direction.