News for hackers. Stuff that matters.

A comment on Khan Academy integrates GPT-4 as every student’s customized tutor. Parent comment

Re:Khan Academy integrates GPT-4 as every student’s customiz... by yamtaddle on Tuesday March 14, 2023 @04:51PM (#35158356)
At least misleading instruction from teachers tends to be of a sort that others also experience, so can later be addressed en masse (e.g. mythologized history, poor-but-common explanations for things like how airplane wings work). Or spreading the same political propaganda that can be found all around, so is pretty easy to spot.

It's not usually that you ask them a question they can't answer, and they just confidently make up something entirely wrong, and possibly also an incorrect explanation that few or no other people have seen.

It's a different kind of wrong. One can write a book addressing common misconceptions picked up in school, from human teachers, and cover much of it. Does that work for lies AIs tell? They could be about almost anything, they could be broadly wrong, they could be subtly wrong, et c. "Show me some things my teachers may have been wrong about" seems to me an easier request to adequately address than "show me some things my AI tutors lied about", because the former tend to cluster around a kind of common space and form an experience with a great deal of overlap between students, while I'm not sure the latter do.

[ Parent | on HN ]
Re:Khan Academy integrates GPT-4 as every student’s customiz... by falcor84 on Wednesday March 15, 2023 @10:32AM (#35168858)
I suppose that this is true. But at least with an AI under your control (as a Khan Academy admin), you can run a query to analyze any unexpected decrease in student performance, find correlations with changes in the keywords that these students received from the AI tutor, isolate the particular misleading AI outputs by running some (possibly automated) experiments, and then once identified, fix the problem systemically by prompt engineering, fine-tuning or output filtering.

What's the analog of that with human teachers, and how long would it take to implement a solution there? Even if you had all the budget in the world, how would you go about making all teachers stop misleading students about how airplane lift works?

[ Parent | on HN ]