Currently AI seem to have 2 weaknesses, it’s quite bad at reasoning and it hallucinates.
Yes, humans screw up reasoning too but at least they try. AI like ChatGPT seem to skip critical thinking altogether - there are examples of ChatGPT contradicting itself multiple times in a single conversation.
Then there are hallucinations. When people don’t know something, most just say they don’t know. AI just can’t seem to help itself but make up complete BS that it confidently try to pass off as facts.
P.S. Frankly I think the 2 might be related. AI can’t “sanity check” it’s own output.
Counterpoint: Why "hire" a known-bad teacher? Is a teacher that is known to give fake information better or worse than none at all?
I don't know the answer to this, but I hope that Khan Academy thought about this and decided that the good of GPT is better than it's bad. Or, they can't know so thats why they're doing a limited study before rolling this out.
How many terrible teachers are allowed to continue to teach after decades of disastrous results?
How many ideologues with no interest in teaching, but every interest in indoctrinating young minds, are tolerated because the alternative is "no teacher" and the class wouldn't run?
How many teachers who would fail state exams, teach, despite relying on answer sheets to be "competent"?
I agree with you that on the top end of education, this is no replacement and at best a supplementary tool. For the poor kid in a bad neighbourhood whose teacher is more interested in "de-colonizing" mathematics than teaching mathematics, this is a Godsend.