Re:LLMs can't self-correct in reasoning tasks, DeepMind stud...
by jppittma on Monday October 09, 2023 @03:39PM
(#37824475)
My blind speculation would be that this property is likely to disappear as more and more text is written by non-thinking agents. Most text on the internet prior to 2022 was written by an agent capable of thinking. That may not be true going forward.
Re:LLMs can't self-correct in reasoning tasks, DeepMind stud...
by init2null on Monday October 09, 2023 @04:04PM
(#37824801)
I've been wondering the same thing. It's possible that older pre-LLM data sources may become indispensable for training.
It would be both amusing and disturbing to see older Usenet, Slashdot, and Reddit conversations turned into rare and valuable resources.
LLMs approach learning differently -- whatever reasoning they do possess is an emergent property that arises as they get better and better at language generation. In other words, unlike humans, LLMs learn to "speak" before they exhibit behaviors that look like logical reasoning. Humans do the opposite.