There's more to it. A top university gives you a stamp that signals "I was smart enough to get in" for the rest of your career. That matters.
If you take only the people who can pass the interview and do well on the job, the ones from top universities/FAANG get more opportunities to interview in the first place.
It's much easier to be a capable person who is associated with a prestigious institution than to be a capable person who isn't.
Hopefully the graduate degree matters more in my case.
If you really want a more in-depth answer to your question, go to https://academia.stackexchange.com/ and see what practicing academics say; this sort of question has surely been asked and answered many times. (I'd dig up some links for you but, not to be unkind, the very first thing you'd better get used to if you hope to reach graduate school is finding stuff out on your own given some hints.)
A better university matters only if you're able to take advantage of what it offers. While the core curriculum is more or less the same for all accredited colleges / universities, a better university gives you access to a larger selection of advanced courses, proportionately more well known instructors, and better labs and facilities. However, the pace of a better university is higher, the hours are much longer, and the competition from fellow students much stiffer than in high school; the transition can be very jarring. Only you know whether you're talented enough and able to put in the hours (the latter can substitute for the former to an extent) to do well enough as an undergrad to make it into a post-graduate program, but understand that the bar is generally pretty high.