News for hackers. Stuff that matters.

A comment on AI Crap – Drew Devault. Parent comment

Re:AI Crap – Drew Devault by lijok on Tuesday August 29, 2023 @10:48AM (#37308751)
That has not been my experience with ChatGPT 4. If I were to ask it about some specialized info about quantum field theory or superconductor tech, I'm sure it would send back nonsense fairly frequently. But if I ask it to explain the difference between median, mean and average, or ask for examples on an architectural pattern, it's seldom incorrect.
[ Parent | on HN ]
Re:AI Crap – Drew Devault by captainbland on Tuesday August 29, 2023 @11:04AM (#37308999)
If you prompt directly about the subject it can be fine but it veers into BS territory more often if you try to get it to synthesise information meaning it's less good with applied examples that are given to it by the user. Not always but the error potential is definitely higher.

The problem then I think is that this means that students will have to prompt "on rails" to get reliable answers. The most curious students who want to stretch their knowledge are likely to cause it to generate falsehoods which are presented confidently and convincingly.

[ Parent | on HN ]
Re:AI Crap – Drew Devault by spion on Tuesday August 29, 2023 @01:12PM (#37310884)
I'm not sure the error potential is much higher than with average teachers, but I guess it depends on the domain. For coding the error rate is lower than a typical teacher right now.

I suspect we'll soon figure out an effective way for LLMs to look-up reliable references to confirm their answers, which should improve the situation drastically. As they are now most LLMs are very barebones. An educational LLM could e.g. connect to the university textbook library and use that to verify its answers.

[ Parent | on HN ]
Re:AI Crap – Drew Devault by captainbland on Tuesday August 29, 2023 @02:43PM (#37312074)
I think the main issue today is its inability to just say "I don't know" when appropriate. A teacher can do that and sometimes that's a better answer than a fabrication. A teacher can also use less certain language when their confidence level is low as appropriate.

I think this is probably a technically solvable problem but doing so has potential 'optics'/marketing issues in terms of reducing the level of confidence it projects which a lot of people associate with competence.

[ Parent | on HN ]