Sure, I've made a good living since then and have never had a hard time getting a job when I wanted one (I've mostly consulted and ran my own businesses), but there are a lot of opportunities closed off to me. I've gotten recruitment letters from Google, among others, based on my Open Source work and a book I wrote a few years back, but I suspect strongly (and with ample knowledge of the Google hiring process and requirements) I wouldn't actually be able to get a job at Google today (maybe three or four years ago, but the "elite university" bar has become far more important these days).
So, if I ever really wanted to try the everyday normal software developer lifestyle...ya know, go to work for about 40 hours each week, eat free food, enjoy subsidized massages, call the ergonomics team when my bottom feels funny in my chair, make enough money to buy big TVs and a Prius but not enough to buy property in the valley, etc. I almost certainly would not have that option. I could contract for any number of small and large businesses, and I have, but being an "employee" with all of the good and bad that comes with that, at any place where I would consider working, is unlikely.
Having those options is nice...at 18 or 19, it's hard to know exactly what one will want out of life. I'm content with the path I've taken, but I'm sure I wouldn't regret having spent my college years at a good school instead of screwing around at community college, either.
If you're bright and independent-minded, the opportunity cost of that 4 years of college is huge. I did very little related to computers in those 4 years - I did get my degree in CS, and I worked on a large volunteer website, and I did a couple projects for my school, but I spent most of my time there as a physics major or trying out various liberal-arts classes. I graduated when I was 24. I have a friend that's 23, and he's currently maintaining 2 open source projects, formerly on another one (LiveJournal), has a good job in a specialized niche, previously had been a startup cofounder that went bust really quick, and has done webmonkey consulting for dozens of different organizations.
So I dunno. As you say, the problem is that it's impossible to tell ahead of time what you want out of life. When I was 20 and entering college, I sorta instinctively knew I wanted to be a startup software engineer (which is what I wanted at 19) yet told myself I wanted to be a theoretical physicist (which is what I wanted at 18) and went to a liberal arts school (which is what I wanted at 17). If it turns out you were wrong when you were young, then you thank your lucky stars that you went to that elite college and have many doors open before you. But if it turns out you were right, then you might be cursing yourself for the lost time spent hedging your bets instead of developing your specialty.
One other thing: doors do slam shut if you choose not to walk through them. I have friends who are in their late 20s or early 30s, straight-A alums of this elite college, and they've had a series of temp jobs because they couldn't decide what they wanted to do with their life. They're finding now that they're on the "no career" career. And while most of them are reasonably happy with their lives, they can't go back to the investment bank or biotech research lab or big software company that they turned down a job from upon graduation.
Maybe I'm missng something, but at least the first couple years of college don't have to be $900/credit plus dorm plus xyz extravaganzas. You're taking mostly required crap like Science credits and English 2. At least that way if you decide college is not for you, you haven't mortgaged your future.