You're arguing that everyone might be good at sports if they had a personal trainer. How is this not a good thing? If you take these models only by the base value they provide, you might be able to eliminate private tutoring from the equation and make performance of students more comparable.
I am far from happy that these models originate from shady for-profit corporations with dubious incentives, but a state actor could just as well train a model for support in education.
> Your response is to rely on opaque knowledge agents produced by the largest and shadiest corporations that humanity has ever known.
This is ironic. My teachers organized a big week about the importance of the personal carbon footprint when I was still in high school (~2015). The personal carbon footprint is criticized for shifting blame for climate change to personal consumer behavior, rather than corporations and was subject of a big advertising campaign from BP [0].
I think you overestimate the average teacher in terms of their ability to think critically and overall skill in their respective subjects.
[0] https://en.wikipedia.org/wiki/Carbon_footprint#Shifting_resp...
Not the OP, but throughout human history, being in the upper percentile for intelligence has been an advantage. The safe bet is that LLMs won't change that.
>opaque knowledge agents
By definition this true for every knowledge agent in history. If Person A is ignorant and Person B offers to teach them, there is always (by definition) an asymmetry. Aristotle was an opaque knowledge agent to Alexander.
I have total control over the model, literally dozens of generation parmaters to play with, literally thousands of open source projects like this
https://github.com/Mihaiii/llm_steer or https://colab.research.google.com/drive/1a-aQvKC9avdZpdyBn4j...
which allow me to jailbreak the model to my hearts content - except I didn't need to do that since Mistral isn't actually aligned anyway!
While you're the one relying on "opaque" knowledge agents, I will have DPO'd my open access model with my custom huge debate evidence dataset to be a better competitive debater than any living human on the planet.
Good luck.
We're outsourcing our sovereignty to a machine blob in a black box, the blob will do the thinking for all of those children.
edit: for the authoritarian tyrants that they are (mass reflexive downvotes)
Your response is to rely on opaque knowledge agents produced by the largest and shadiest corporations that humanity has ever known. Your lack of paranoia is a severe vulnerability. Good luck.