Some of the densest cities on the planet are on islands. San Francisco is on a peninsula. It is a low density city with tons of infill potential. That densification potential, together with slow population growth in the developed world, makes the problem perfectly tenable. (That said, homeowners would lose money. So not holding my breath.)

In the short term they would. And in the long term, they would gain wealth but lose their iron grip on the city. Never underestimate the seductiveness of power. As long as San Francisco’s renters believe density is impossible or will price them out, homeowners (specifically, about half a dozen families) will retain their lock on the city.

You mean, "for very specific political and cultural reasons". There's nothing in San Francisco's geography that most world cities haven't solved.