Good on you, but you're also kidding yourself. First impressions matter---and hearing someone went to MIT or Harvard or works at GoogFace immediately implies that they passed their rigorous vetting. In the absence of more information and the time to process it, it's a pretty strong signal.

It also matters socially. I went to an Ivy League grad school and a no-name undergrad, so I can provide a point of anecdata that people treat you differently.