People also routinely fail to reason, even programmers often write "obvious" logic bugs they don't notice until it gives an unexpected result at which point it's obvious to them. So both humans and AI don't always reason. But humans reason much better.
I myself have observed ChatGPT 4 solving novel problems I invented to my personal satisfaction well enough to say that it seems to have a rudimentary ability to sometimes show abilities we would typically call reasoning, but only at the level of a child. The issue isn't that it is supposed to reason perfectly or that humans reason perfectly, the issue is that it doesn't reason well enough to succeed at completing many kinds of tasks we would like it to succeed at. Please note that nobody expects it to reason perfectly. "Prove Fermat's last theorem in a rigorous way. Produce a proof that can be checked by Coq, Isabelle, Mizar, or HOL in a format supported directly by any of them" is arguably a request that includes nothing but reasoning and writing code. But we would not expect even Wiles to be able to complete it, and Wiles has actually proved Fermat's last theorem.
So we have an idea of reasoning as completing certain types of tasks successfully, and today humans can do it and AI can't.
Today, it fails badly at tasks that require reasoning. A simple example: https://chatgpt.com/share/da95843e-218a-4d69-a161-6aa2d7a3c9...
The issue is that humans can see its answer is wrong and its "reasoning" is wrong.
The issue isn't that it never reasons correctly. It's that it doesn't do so often enough or well enough, and it doesn't complete tasks we expect humans to complete, and it doesn't always notice when it is printing something outrageously wrong and illogical.
It notices sometimes, it engages in elementary rudimentary guesswork sometimes, but just not often enough or well enough.
> The issue is that humans can see its answer is wrong and its "reasoning" is wrong.
I've noticed with LLMs that they're more likely to come to the wrong conclusion if you prime them in that manner. In this case, you posed the follow-up question as "Will <incorrect conclusion> always be true?" As a result, it's primed to try to prove that incorrect conclusion.
(That said, ChatGPT further did not answer the posed question, as it also changed "difference" -> "absolute difference"; in fact, the difference will alternate between increasing and decreasing, while the absolute difference is strictly increasing.)
That's why I think of GPT3+ as "subhuman AGI," personally.
I think until we know the answer to this, we can't make predictions about how to build true AGI.
Rarely, actually.
More generally humans use all kind of inferences where problem at hand is intertwined with all other attention points that is occupying the mental load of the person. Giving a topic full mental attention and finding a path through pure deduction about a circumscribed subject is a rarity, even if you consider only those situations that require any conscious attention at all to perform some action before moving on.
nope. most humans fall in various traps such as pattern recognition, confirmation bias, and many others instead of relying on deductive analysis. Even scientists fail at being rigorous.
Just our visual object recognition is immensely powerful and far beyond and current AI. A simple task like walking to the fridge requires a ton of pattern recognition and spatial reasoning. Recognizing people's moods/predicting behaviors is also incredibly involved imo.
Ive said this many times but perhaps we should focus on achieving dog level intelligence first before we start worrying about human level AGI.
That's why nobody has gotten any traction selling access to AIs for $20 a month whereas selling access to mouse labor is such a thriving business.
Just because LLMs are useful, it doesn't mean they exhibit more intelligence than a mouse. A mouse probably also doesn't reason about anything, but it is an agent capable of independent behavior, something that is still very far removed from current AI models.
OK, as long as we're are being humble, how about we refrain from confidently proclaiming that there is a mouse level and a dog level that AI hasn't reached yet and that researchers will have to spend a long time getting past, so there's plenty of time before we have to worry about the possibility of AI's becoming dangerous or transformative to society?
That's a point you'll likely have to revisit pretty soon. Radiology, for instance, probably won't exist as a profession 20-30 years from now. Captchas are already pretty much done for.
Lastly, check out the ARC challenge or any other spatial reasoning tests for AI. Humans get ~80% on these challenges whereas the best AI is still at 25%
Also, are you familiar with this study? What are your thoughts on it? https://www.esmo.org/newsroom/press-and-media-hub/esmo-media... Seems like a valid case where AI is competitive with skilled humans at object/image recognition.
https://lab42.global/arcathon/leaderboard/
https://openreview.net/forum?id=E8m8oySvPJ
As to the study, I have the same objection as the radiology one. This isnt about object recognition and certainly not spatial reasoning, its the ability to predict cancer based on presence of visual features.
The "object recognition" part of this is super simple. Its a single, mostly 2D object in more or less the same angle, and the AI is trained on detecting just this.
And yet it outperforms human dermatologists.
For humans, it is emergent. But when we reason about reason, we invent special sauce.
If we build our theories of reason into our models, they achieve the strengths and limitations of our models.
If we don't, we're limited by the pace of evolution, because we don't have enough connections in our graph.
So I think we'll have something immediately more useful if we embed ALU special instructions into a neural network.
However, humans have the ability to reason about things (whether most people use this ability is a different question). So then we must ask the question: is this ability just a more advanced form of probabilistic pattern matching, or is it a different architecture altogether? Will current AI models be able to develop this ability, or will we need new models?
For transformer-based LLMs, and most LLMs there's an obvious class of problems that they cannot solve. LLMs generally perform bounded computation per token, so they cannot reason about computational problems that are more than linearly complex, for a sufficiently large input instance. If you have a back-and-forth (many shot) your LLM can possibly utilize the context as state to solve harder problems, up to the context window, of course.
The fact that so many people can’t see the fundamental differences of an LLM and human intelligence reminds me of back when the very early computer scientists thought they could model the entirety of nature by reducing every “component” to a numeric value and compute it as “transfer of energy”.
Quite literally they did the same thing: They had a new toy (very advanced computation machines) and forced all of nature to “fit” within it. It also ended in failure, obviously. Not because nature or ecosystems (as it was coined) are “magic” but because grossly oversimplifying reality to fit desired models is a fool’s errand.
I can’t judge if this is true, because I don’t know transformers well, but if it is, it unravels an intuitive thought I’ve never been able to articulate about not only LLMs, but possibly all pattern matching and the human analog of System 1 thinking.
Another fuzzy way of saying this is there’s something irreducible about complexity that can’t be pattern matched by any bounded heuristic – that it’s wishful thinking to assume historical data contains hidden higher-level patterns that unlock magical shortcuts to novel problems.
In the right context, why not? You rely on this everyday to navigate the world with more facility than a newborn.
Have you heard about the different formal notions of complexity and especially Kolmogorov complexity?
Sounds like the AGI argument trap: They're not able to reason, but we can't succintly define what it is.
I don't come with a reasoning chip. Whatever I call reasoning happens as a byproduct of my neural process.
I do think that the combination of a transformer network and calls to customized reasoning chips (systems that search and deduce answers, like Wolfram Alpha or logic/proof systems) may be a short-stop to something that can perform reason and execution of actions better than humans, but is not AGI.