LLMs can't self-correct in reasoning tasks, DeepMind study finds

Re:LLMs can't self-correct in reasoning tasks, DeepMind stud... by og_kalu on Monday October 09, 2023 @03:47PM (#37824587)

By far one of the most fascinating things to do is look up up GPT-2, 3 threads and see predictions, extrapolations of future abilities. They're of course hilariously wrong but the best bit is that the vast majority of people see to be making the exact same mistake.

"GPT-a has no real x so of course it can't do y" only for y to not be a problem in a future iteration.

It's like people just can't grasp that a predictor being unable to do x isn't an indictment on prediction. The predictor is just not good enough yet.

That GPT-2 could not predict coherent passages did not mean that you couldn't predict your way to coherency. GPT-2 was simply not a strong enough predictor.

That GPT-3 could not predict valid chess games did not mean that you couldn't predict your way to playing a chess game. GPT-3 was simply not a strong enough predictor.

[ Parent | on HN ]

Re:LLMs can't self-correct in reasoning tasks, DeepMind stud... by roflyear on Monday October 09, 2023 @05:24PM (#37825708)

This is a logical fallacy itself, no? Correct me if I'm wrong, but it seems like you're making the argument for infinite capabilities with LLMs. This is almost certainly not true.

[ Parent | on HN ]

Re:LLMs can't self-correct in reasoning tasks, DeepMind stud... by og_kalu on Monday October 09, 2023 @05:57PM (#37825981)

I'm not making the argument for infinite capabilities. The main point is that the idea that you can determine this stop from looking at current iterations is a fallacy. It's no more evidence than random chance.

The second point is to understand that any task, any task at all even if it's not necessarily the most efficient or performant way can be expressed as a prediction task and sit along for the ride.

If your model of the world is good enough, there's nothing occurring in the world you can't predict. Prediction itself is not the weak point here.

[ Parent | on HN ]

Re:LLMs can't self-correct in reasoning tasks, DeepMind stud... by roflyear on Tuesday October 10, 2023 @12:25PM (#37834205)

I understand your perspective that with a sufficiently accurate model of the world, anything can be predicted. But I disagree, while framing all tasks as prediction problems is an interesting perspective (I haven't heard this explicitly stated before), there are inherent uncertainties/data gaps/etc.. in some systems that might pose challenges even to the most advanced predictors. I think your argument relies on there being perfect data - which doesn't exist, and will never exist.

It's not solely about the predictor's strength but also about the intrinsic predictability of certain phenomena.

[ Parent | on HN ]