Yes, I would agree with your statement that what matters is who you're trying to impress. If you want to impress a graduate admissions committee, they are going to prefer certain schools for various reasons. When I think of an "elite" school, I am thinking in a more traditional sense. You aren't networking with the offspring of the people who are running countries and industries, the overall reputation and the type of students who attend are what separates "target" and "elite" schools. Outside of job interviews you don't really have the option to go through your CV and explain your research and related publications. Telling someone that you went to school in New Haven is much more impressive than telling them you went to CMU for this reason. I doubt Mark Zuckerberg would have met an Eduardo Saverin at CMU or UIUC.

Ah I apologize for misunderstanding -- I agree with this point quite well