News for hackers. Stuff that matters.

A comment on LLMs can't self-correct in reasoning tasks, DeepMind study finds. Parent comment

Re:LLMs can't self-correct in reasoning tasks, DeepMind stud... by godelski on Monday October 09, 2023 @05:06PM (#37825481)
I love this example! I've been using a different one and GPT has been getting better at it (unconvinced they aren't training on my reporting that it gets the wrong answer. Or that I've posted this question dozens of times here and some have gotten pretty high up. But it's been a year now and we're still in stochastic phase). The question I ask is variants of "Which weighs more, a pound of feathers or a kilogram of bricks?" Similar to yours it is just a slight variant on a standard logic puzzle.

For those wondering why ask these questions, we're testing for overfitting. Yes, you can overfit even if your training and validation curves don't diverge. If the model fails these questions, they've clearly overfit. But the solution space is very large and complex, so overfitting in one regime doesn't mean it didn't underfit another.

GPT 3.5: https://chat.openai.com/share/328cc39d-7fb3-4726-92a2-29437c...

LLaMA 2 70B chat: https://hf.co/chat/r/KX5H3P2

Falcon 180B Chat: https://hf.co/chat/r/V8XxdYh

None of these models can consistently get the answer right. GPT used to actually explain to me the difference between a pound and kilogram (correctly) and then use that answer to justify why they are the same. Such an answer is very clearly a demonstration of a lack of understanding as it isn't even remotely self consistent.

LLMs are powerful and amazing technologies. But we can also critique them. Hyping up models hinders the ability to improve them because every thing (LLMs, humans, governments, whatever) has limitations and is worthy of critique. But criticism isn't saying something is garbage and too many people confuse this.

[ Parent | on HN ]
Re:LLMs can't self-correct in reasoning tasks, DeepMind stud... by jfim on Monday October 09, 2023 @05:31PM (#37825789)
Agreed, there's definitely a lot of value in LLMs even if they don't really have an understanding of the world.
[ Parent | on HN ]
Re:LLMs can't self-correct in reasoning tasks, DeepMind stud... by godelski on Monday October 09, 2023 @08:21PM (#37827119)
It is quite interesting to me that people latch onto the world model so much as if it is highly significant or a lack of a world model invalidates LLMs. Personally I still believe they're magical (in the proverbial sense) and amazing tools even though I also believe they are just complex stochastic machines that have very little understanding of the things they are doing. That just tells me how cool and amazing that statistics is. I honestly don't know anyone that thinks LLMs are useless things, despite knowing a large number of people that are highly critical of them. But those are very different metrics. I don't know why we can't have nuance in things, why they need to either be a gift from god or a pile of shit.
[ Parent | on HN ]
Re:LLMs can't self-correct in reasoning tasks, DeepMind stud... by spdustin on Tuesday October 10, 2023 @08:43AM (#37831324)
I was pleased to see GPT-4, combined with my custom ChatGPT instructions, had no problem with this: https://chat.openai.com/share/726d5c7b-761f-462e-8a6a-61904a...
[ Parent | on HN ]
Re:LLMs can't self-correct in reasoning tasks, DeepMind stud... by godelski on Wednesday October 11, 2023 @12:57PM (#37847028)
You should refer to this comment I made[0], as your prompt is spoiling the problem. It is far too specific. Hacking the answer in is not a means of getting correct answers. It won't scale and won't generalize.

[0] https://news.ycombinator.com/item?id=37825566

[ Parent | on HN ]