I'm not sure the error potential is much higher than with average teachers, but I guess it depends on the domain. For coding the error rate is lower than a typical teacher right now.
I suspect we'll soon figure out an effective way for LLMs to look-up reliable references to confirm their answers, which should improve the situation drastically. As they are now most LLMs are very barebones. An educational LLM could e.g. connect to the university textbook library and use that to verify its answers.
Re:AI Crap – Drew Devault
by captainbland on Tuesday August 29, 2023 @02:43PM
(#37312074)
I think the main issue today is its inability to just say "I don't know" when appropriate. A teacher can do that and sometimes that's a better answer than a fabrication. A teacher can also use less certain language when their confidence level is low as appropriate.
I think this is probably a technically solvable problem but doing so has potential 'optics'/marketing issues in terms of reducing the level of confidence it projects which a lot of people associate with competence.
The problem then I think is that this means that students will have to prompt "on rails" to get reliable answers. The most curious students who want to stretch their knowledge are likely to cause it to generate falsehoods which are presented confidently and convincingly.