In today's system if you can afford university with no debt you're very well off and it's probably a wash since the funds are coming largely out of your taxes (ie just about everyone that works in US tech and comments on this forum). The biggest winners are middle class families that can prepare their children for college but can't pay for it; but again, helping one group doesn't mean that it's no longer possible to help another group.
I can't see how restricting higher education back to the days when the hyper rich gathered to circle jerk about how much of the classics they've memorized helps in anyway (other than give all of us already educated in the workforce a lovely gate against the next generation).
Meanwhile, if university education really is mostly a vacation for wealthy young adults, then funding it is regressive, because no matter how directly accessible you make it financially, it's still tilted towards the wealthy because they have the means to forego 4 years of income to attend, and because they're better prepared to apply and take advantage of it in the first place.
A dollar spent on accessible college is just a dollar that can't be spent else where not necessarily a dollar taken from another services. Despite what some people might lead you to believe, taxes can be raised to pay for things (usually disproportionately affecting those wealthy people you're so concerned about getting a handout).
[0]https://cew.georgetown.edu/cew-reports/the-college-payoff/
You can disagree, and then make your case, but you have to make it on the merits, not with mic-drop logic.
Note that my side already has a highly instructive example to argue from: the mortgage interest tax deduction, which turned out to be expensive, distortive, and highly regressive.
As to being elitist: subsidizing college is a handout to people who are already pretty well off. The people who need help the most aren’t going to college. I’d very much like to see colleges defunded and that money redirected to social services for kids in high-need communities.