LLMs can't self-correct in reasoning tasks, DeepMind study finds

Re:LLMs can't self-correct in reasoning tasks, DeepMind stud... by telotortium on Monday October 09, 2023 @04:11PM (#37824884)

> This makes it much more difficult to learn how to reason about particular data that appears inside the token, because it does not ever receive that information.

More difficult, but it shouldn't be impossible. The Chinese managed to make rhyming dictionaries over 1000 years ago despite using a non-phonetic writing system. More relevantly, GPT-4 has definitely ingested both regular and rhyming dictionaries.

[ Parent | on HN ]

Re:LLMs can't self-correct in reasoning tasks, DeepMind stud... by wilg on Monday October 09, 2023 @04:32PM (#37825141)

I agree it should be possible.

[ Parent | on HN ]

Re:LLMs can't self-correct in reasoning tasks, DeepMind stud... by vidarh on Tuesday October 10, 2023 @04:43AM (#37829792)

Possible, certainly, but given the difficulties it has with it, it appears it likely requires more training. Unlike with Chinese, the LLMs will not benefit from extensive training data mapping its tokens to and from sounds or rhyming with other words, but need to derive during training from data that rarely mentions these tokens at all.

[ Parent | on HN ]