People, by and large, need artificial encouragement to help them do what they want to do in the first place, but lack the willpower or procrastinate too much. People subscribe to classes so they can learn in a formal setting material they could perfectly well study on their own before, but didn't. People take out gym membership so they have no excuse not to go. People ask relatives and friends to help force them prepare for something important. You may be a champion of iron will (though I doubt it), but most people aren't, and they're still quite capable of learning nevertheless.
In the recent ML/AI course offerings from Stanford, the exercises were a joke - they were painfully easy. But if they didn't have graded exercises with deadlines, probably five times fewer people would have finished the course. In fact, similar courses with entire lectures and homework have been posted on the web for years, with little engagement, while this new offering, with a formal setting that graded you on exercises, has seen tremendous success.
Your rhetorical question is exceedingly naive.
The contrived setting of the classroom is much better than an online setting. This may change as technology gets better. Right now, online is not a better format than face-to-face. Online might be the most cost effective format but it isn't the pedagogical optimum.
I don't think you've made a compelling argument on that score. It is precisely because life is complicated that lower cost and more flexible method might be considered optimum.
There are however a lot of students who see academics as something of a game and measure their achievement by score, the fact that they may learn a few things along the way is secondary to their desire to be able to say "hey , look at me I'm a high achiever please pay me lots of money". These people would probably be lost without a grading system of some kind and a lot of them are actually very smart people who probably should be at university.
And while this more of a philosophical point than anything, I think that this form of competitiveness ("look, I'm better than you") is a major problem of today's society. Education should teach to work together, not against each other.
Experiments in reducing the grade-centricness of universities have mostly foundered on employer objections. UC Santa Cruz for a while was famous for giving narrative evaluations instead of grades. Clases were formally graded pass/fail to indicate whether you met the minimum requirements to move on, and then in either case the professor wrote a one-page or so explanation of what you did well and poorly in the class, some advice, etc. In a sense the evaluation was still quite rigorous, just not boiled down to a single letter.
Companies hated this because they wanted to know which students had the 4.0 GPAs, and which had the 2.0 GPAs: a string of passes plus a stack of narrative reports didn't fit their evaluation model. So UCSC moved to allowing students who wanted to receive a grade along with the evaluation to opt into an A/B/C/D/F grading model alongside the narrative evaluation; eventually they made that mandatory; and finally just dropped the narrative evaluations.
You would probably just have recruiters who would grade candidates on the basis of how many times the word "excellent" appeared on the report or similar, no doubt startups would pop up offering machine learning type solutions to these problems as well.
Not to mention it would be easier to corrupt the system as there would be more plausible deniability if a professor simply "forgot" to include some negative factors on the report.
Much easier just to filter numerically based on grade. Also most of the smart people who do enjoy the learning as an end in itself should know the material well enough to do well in an exam anyway and just view the process of revising for the exam and tweaking their coursework for good marks as "playing the game".
I'm not convinced that academics are better qualified to predict the needs of society regards knowledge and education any more than the free market is.
Regards to helping students evaluate themselves , there is no reason that a professor cannot provide feedback in addition to assigning grades indeed I'm sure many would be happy to and it's more likely a case that many students are simply happy to get their grade and move on rather than evaluate themselves.
On the contrary, I'm pretty sure the free market is the last thing qualified for that. It will always be a race to the bottom, and that's not something education should be.
Would Stackoverflow.com have the participation it has if it didn't have badges, numbers, and karma? I think your perspective ignores a great deal of research on games and human behavior.
And that's exactly the problem with the entire system. We need other ways to give incentives and motivation. The point is that plain numbered grades suck at giving motivation, they suck at actually judging the performance and/or understanding of a student and they suck at being anything but a completely arbitrary method of weeding out people who are unable to conform to the contrived requirements of an artificial testing environments, whether they are actually intelligent or not (and being a slow learner doesn't mean someone is dumb, another thing this system sucks at).
In my opinion, if your not at the university to learn first and foremost, then what the fuck are you doing at the university in the first place?