Chain-of-Thought Prompting Elicits Reasoning in Large Language Models https://arxiv.org/pdf/2501.19201
It effectively treats “reasoning” as the ability to generate intermediate steps leading to a correct conclusion.
Now, is this valid reasoning? Well, depends on the claim and the definition of reasoning.
When someone just says AI can’t reason, I could argue for or against that depending on the specifics. It’s not enough to just say yes or no.
>> It effectively treats “reasoning” as the ability to generate intermediate steps leading to a correct conclusion.
Is "effectively" the same as "pretty precise" as per your previous comment? I don't see that because I searched the paper for all occurrences of "reasoning" and noticed two things: first that while the term is used to saturation there is no attempt to define it even informally, let alone precisely; and second that I could have replaced "reasoning" with any buzzword of the day and it would not change the impact of the paper. As far as I can tell the paper uses "reasoning" just because it happens to be what's currently trending in LLM circles.
And still of course no attempt to engage with the common understanding of reasoning I discuss above, or any hint that the authors are aware of it.
Sorry to be harsh, but you promised "examples that go back 50 years" and this is the kind of thing I've seen consistently in the last 15 or so.
Research papers relating to reasoning approach and define it in many ways but crucially, the good ones offer a testable claim.
Simply saying “models can’t reason” is ambiguous to the point of being unanswerable.
For example, if I throw a bunch of sticks in the air and look at their patterns to divine the future- can I call that "mathematics" just because nobody has a "consensus definition of mathematics that can be applied as a yes/no test"? Can I just call anything I like mathematics and nobody can tell me it's wrong because ... no definition?
We, as a civilisation, have studied both formal and informal reasoning since at least a couple thousand years go, starting with Aristotle and his syllogisms (a formalisation of rigorous arguments) and continuing through the years with such figures as Leibniz, Boole, Bayes, Frege, Pierce, Quine, Russel, Godel, Turing, etc etc. There are entire research disciplines that are dedicated to the study of reasoning: philosophy, computer science, and, of course, all of mathematics itself. In AI research reasoning is a major topic studied by fields like automated theorem proving, planning and scheduling, program verification and model checking, etc, everything one finds in Russel & Norvig really. It is only in machine learning circles that reasoning seems to be such a big mystery that nobody can agree what it is; and in discussions on the internet about whether LLMs reason or not.
And it should be clear that never in the history of human civilisation did "reasoning" mean "predict the most likely answer according to some training corpus".