Re:Re-Evaluating GPT-4's Bar Exam Performance
by lupire on Monday June 03, 2024 @07:27AM
(#40561606)
How do yo debug its hallucination misinformation via voice interface while you commute?
Re:Re-Evaluating GPT-4's Bar Exam Performance
by fnordpiglet on Monday June 03, 2024 @03:20PM
(#40566471)
I just use my memory and verify later. Unlike a LLM I have persistent long term durable storage of knowledge. Typically I can pretty easily pick out a hallucination though because there’s often a very clear inconsistency or logical leap that is non sense.
Automatic proof generation is a massive open problem in all of computer science and not close to be solved. It’s true LLMs aren’t great at it and more is required for example as with the geometry system Deepmind progresses on.
On the other hand they can be very useful to explain concepts and allow interactive questioning to drill down and help build understanding of complex mathematical concepts, all during a morning commute via the voice interface.