Yes. What do you think people bidding for ultra-lux housing do if they don’t have supply?
In New York, the ultra-wealthy have penthouses. In San Francisco, they have rows of townhouses. Which one do you think displaces more affordable housing?
Start here: San Francisco, for very specific geographic and regional reasons, will not have sufficient housing in the next century, and it will continue to have growing homeless populations. Solve for a 50-year horizon.
Some of the densest cities on the planet are on islands. San Francisco is on a peninsula. It is a low density city with tons of infill potential. That densification potential, together with slow population growth in the developed world, makes the problem perfectly tenable. (That said, homeowners would lose money. So not holding my breath.)
In the short term they would. And in the long term, they would gain wealth but lose their iron grip on the city. Never underestimate the seductiveness of power. As long as San Francisco’s renters believe density is impossible or will price them out, homeowners (specifically, about half a dozen families) will retain their lock on the city.
You mean, "for very specific political and cultural reasons". There's nothing in San Francisco's geography that most world cities haven't solved.
You said yourself that the new construction had slightly higher pricing meaning your unit was in the same basic market as the new construction and also implies you likely were not shopping at "the bottom of the market". Do you think you would have been able to negotiate your rent down if the new building was in an entirely different class and was charging 2x or 3x your rent?