Well, this is what free market capitalism is for. If it weren't for the loads of regulation whole neighborhoods in SF would have been replaced with high-density housing a decade ago.

However, California has decided that "housing as investment" supersedes all else. It's not that there's no viable alternatives: this is by design.