When you ask why Manhattan is expensive even with increased supply, you ignore demand.
The theory that increased density in itself drives up the value of housing is popular among anti-housing activists. I've never heard an economist who believes it.
If true, it would be an incredibly simple way to generate enormous amounts of money to simply building super dense neighborhoods. Since I don't see anyone getting rich off that, I have strong doubts about that model.
People don't move to where they can find expensive housing - people move to where they can find good jobs. Doubling SF housing wouldn't double the number of good jobs there.
Thought experiment: What do you think doubling the number of high paying jobs do to SF rent? Or, alternatively, halving the available housing?
Thought experiment: Let's say we do massive high density development in SF. A bunch of people move in, but now that SF is denser it's more desirable, so demand has increased as well, which drives rents right back up. Now we're stuck where we were before: high rents.
Having said that, I still support high density development because I think it leads to better living (and is better for the environment usually). I just am not optimistic that it will actually lower rents.