Why would it be just "poor non-graduates" subsidising "middle class kids"?

An potential extra tax burden can be allocated from wherever the government sees fit. Like a digital services tax or closing loopholes that are conveniently left open for global corporations.

Do you worry that the healthy poor people are subsidising the middle class sick ones through the NHS? Or that poor careful drivers are subsidising the cost of accidents for the middle class careless ones via their insurance company?

Everyone benefits from the NHS.

The moral hazard is non graduates (including pensioners) subsiding graduates.

Though it's less direct than the NHS benefit, I think it's a pretty good argument that everyone benefits from an educated society, even if you can't point at every single person with a degree and say they're contributing to the greater good.

Educated people design infrastructure, products, services, (public and private) that affects every aspect of everyone's life.

I feel like this would be unfair on non-graduates, but only because government support for alternative career paths is so poor (e.g. apprenticeship schemes are a joke in this country). But really nobody should be entering working life in debt.