Re:Are Emergent Abilities of Large Language Models a Mirage?
by Mike_12345 on Monday May 01, 2023 @04:24PM
(#35778641)
All you have shown here is that its arithmetic reasoning is flawed, not that it cannot reason at all.
"Reasoning" is a much broader term than you think it is.
Re:Are Emergent Abilities of Large Language Models a Mirage?
by flangola7 on Tuesday May 02, 2023 @08:42AM
(#35786535)
Many humans would not be able to solve that problem, especially those that are younger or have lower IQs, and obviously those that have not not been taught multiplication. That doesn't mean they are devoid of all reasoning ability.
Furthermore GPT-4 is able to multiple slightly shorter pairs of numbers, and experiments on smaller neural nets, such as one using a dataset consisting of 20% of all combinations of two four digit numbers, quickly learn to generalize to successfully multiply any four digit pair even though they aren't in the training set.
Try again, give an answer that actually works this time.
Any human who has learned multiplication can do this. AFAIU, LLMs cannot unless the computation exists within the training set. They have zero arithmetic reasoning capability.
I just asked ChatGPT: "19191920 multipled by 10292111772"
It said:
--------------
----------------------------------This is completely wrong. It is not doing arithmetic, and it is not capable doing arithmetic.