To me, it's a scenario of pick your poison - continue to be unaffordable because society is not willing to accept the consequences of its failures, or eat the consequences for a brighter future. In cities like SF, I don't see a world where it will ever get affordable if they insist that everyone needs to live in 1000+ sqft apartments, unless you plan to pack it to the gills with ultra-tall skyscrapers (never happening) or it becomes so undesirable to live there that people just stop moving there (unlikely, but given the rampant human rights violations it commits everyday to its denizens... well maybe that's the most likely scenario).
Aside, Americans don't know what small apartments even mean. The presence of super tiny Tokyo apartments is a great thing IMO, it means that people can still live centrally even if they aren't rich. I think a lot of people would happily accept a well-designed 200 sqft apartment over a 2-hour commute, if society gave them that option, since Tokyo residents do that en masse everyday.
And yet there are plenty of single family homes and 2500+ sqft apartments if you want that. Just in the USA, if you want to live small - the government forcibly tells you that you’re not welcome here.