My wife worked for a startup in SF next to a cool bar and a cool coffee shop. When you left work, a couple of your friends/coworkers would often be in that bar or coffeeshop. So you'd stop in, and start chatting. Often, you'd chat about work, and come up with new ideas. They might get excited enough to go back into work to hash it out a bit more - at least they'd have something they were charged up about.
At the time, I was working for Sun Micro in Fremont, a spiritually crushing place. On occasion, we'd all agree to get together for drinks at the W nearby. But it took planning, arranging, getting into your car, driving over there... not at all spontaneous. And so we didn't get to know each other as well, and didn't have any fun.
That said - your post is missing one really important thing: San Francisco and Berkeley have a vibe that is very conducive to creativity, and even parts of (admittedly suburban) Palo Alto aren't so bad. This is why I drove a horrendous hour and fifteen minutes to work every day. It's also why I finally got so fed up with the commute that I faced a choice: quit sun, or move to Fremont. I quit.
But hey, would you rather live in SF and be an hour from the valley, or in Munich and be a trans atlantic flight away? One poster called SF a "second rate new york" in a previous thread (obviously not getting it). San Francisco is not a big city like New York, so don't go looking for Manhattan. Look for a small (surprisingly small to some, considering how many comparisons to NY I hear) but very entertaining city with a terrific vibe and a ton of technical and creative people to hang with.
I remember trying to make the more than 1 hour drive from there to SF, and then back again... just to spend a few hours in a the nice city. Yes, I'd rather live in Munich.
This is a huge topic and it affects the entire USA. Our whole country is stupid automobile sprawl. "The greatest mis-allocation of resources in the history of mankind."
Hang out in the outer suburbs of Paris some time, and you'll see what I mean - not a big improvement over the outskirts of SF or LA (actually, don't - vacation time in the US is precious, and you definitely don't want to waste it on that).
There are two possible future outcomes I can think of.
1. We have an oil crisis, and cities return to the way they used to be, over the course of time.
2. Alternative energies gracefully take the place of oil, there is no crisis, and things continue as they have been.
Oddly, the crisis outcome is more appealing.
You're kidding, right? Silicon Valley has the best year round cycling anywhere, and some of the best cycling, period. Maybe Switzerland in the summer is better, but that's about it (and it frequently rains in the summer in Switzerland). I lived in Silicon Valley without a car for 7 years. It's flat, and by drawing a 16 mile radius around my apartment in Sunnyvale, I can hit nearly every important tech startup of any consequence.
The quality of life in Silicon Valley sucks if your idea of fun is visiting bars, going to rock concerts, or top quality museums. For people for whom outdoor life is important, Silicon Valley is great. I regularly run into famous outdoorsy types (Brian Robinson, Jacquie Phelan, Jobst Brandt, Tom Ritchey) just by riding around in the area. On my cycling trips around here, I've met the former VP of IT at Adaptec, the VP of Engineering at Portal Player, etc.
And much as you might want it to be true, Fremont isn't Silicon Valley. Neither is Redwood City.
I lived in Fremont for a couple of years, and I can't tell you how much I hated it. Urban sprawl. Nothing to do.