Wait, what? Are you sure you're reaching the right conclusion? It sounds like your anecdote supports the opposite once you stop looking inter-your-small-group, and start comparing groups. Have you compared median wage data for your elite university versus median wage data for the country you are in? I suspect the difference is noticeable.

It appears you are making the mistake of comparing a group of successful entrepreneurs, failed entrepreneurs, and people who went to work at Goog and McKinsey with each-other, but you really need to compare that group to people who couldn't get a job at Google or McKinsey at all. Obviously life is not nearly that uni-dimensional, but I'd wager your median "all really smart" colleague is wildly beating the median person, even if some of them are broke.