LLMs can't self-correct in reasoning tasks, DeepMind study finds

Re:LLMs can't self-correct in reasoning tasks, DeepMind stud... by Jensson on Monday October 09, 2023 @04:20PM (#37825002)

Well, I am trying to explain why your original meme post "it just reads tokens, it can't do that" is wrong. That meme is tiring, and the most tiring with trying to fight such memes is that so many comes and repeats it and it is very hard to change their mind.

But yeah, I should have been more clear to target that meme so that you didn't feel it was about you. It is easy to accidentally make things a bit too personal.

[ Parent | on HN ]

Re:LLMs can't self-correct in reasoning tasks, DeepMind stud... by wilg on Monday October 09, 2023 @04:31PM (#37825129)

OK, thanks.

I'm not claiming its impossible for an LLM to learn this, just that it appears to be much harder (perhaps by requiring more data) when the task involves fighting against tokenization. For example, it can base64 encode your phrase "What was once dormant may soon angrily awake" correctly, and it can reverse the letters correctly. But it fails to do the same on the example text from the OpenAI tokenizer.

[ Parent | on HN ]