I never thought about it. I considered ETH Zurich for a while because Niklaus Wirth is a legend in language development, but without any real idea of how it measured up as a school. In the end I stayed in Norway for university. I don't regret it, per se, but understanding the difference more later was a very strong eye opener for me when it comes to how much of social stratification is not really down to money but more broadly background and expectations.
It takes a very specific background before elite universities will even be on your radar, and a subset of that for you to believe it to be possible, and a subset of that for you to see it as the norm and what is expected of you. And then that needs to intersect with the understanding of your family to either pay or put in the effort to look for other ways of getting in.
Some portion of those will not be rich, sure. There are lots of options. Most top universities are good about grants and bursaries. But proportionally, a far larger proportion of children of wealthy families will go to top universities.
And this is a large part (but not all) of what "privilege" is really about: To grow up in an environment that teaches you to expect certain things that is not even on the radar of others.
This presupposes that you are already in a social position that allows your intelligence to be recognised and that puts you in a position to even apply, or know that applying is an option.