I used to ride a recumbent to UCLA now and then because I had a death wish, and it shocked me that it took 25 minutes and yet people spent 60 minutes in traffic in their cars every day just to get across the 405.
You're right, though, those cities are mostly lost causes, more for the attitudes present in them than the infrastructure. Doesn't mean we can't try to make any new ones better though.
Riding a recumbent to UCLA from 16th is hardcore. Everyone I knew that rode in west LA got hit at least once, if not more.
My brother lived off Geary for a few years and went to a few public meetings on the proposed subway line they want to put under it. He was interested in the idea, as the N-Judah is a methadone clinic (last he was there a few years ago, caveats apply). He never saw such NIMYism and such tight resistance by the property owners on Geary to any improvements whatsoever.
You don't just ease regulations or really do anything 'big' in any city. Palms must be greased, and if you do not it is a horrific process that takes at least a decade.
I'm not saying that you are wrong about the free market. However, my brother thought that the most fantastic thing about the new Star Trek movies (esp. the new Wrath of Khan) was not the warp-drive, nor the aliens, but the very tall building in SF. Even in 400 years, there is no way the planning boards would allow such things.
As someone who lived in West LA for years and saw the new line come in, I'll stop you right there. Cities like LA, Mexico-City, San Jose, etc. are not well designed and trying to re-invent them is going to take one hell of an earthquake.