Human knowledge is ultimately embodied in physical experiences, it is not merely spitting out words. How can an LLM know the taste of a mango? Ability to regurgitate previously written text about mangos is neither knowledge nor intelligence. To know something, one must experience it.
Even mathematicians "experience" their formulas and proofs in an ineffable way, they don't just produce text. There's an "aha" moment when learners "get it" and they experience knowledge and are then able to produce correct answers and generate new knowledge and discoveries. LLMs just generate text, and have yet to produce any significant new discovery in any field.
LLMs are like fake wine sommeliers who have read a lot of lingo about wine and can speak convincingly enough to fool an amateur but would fail a simple taste test. We would say such a person doesn't know what he's talking about.
People who think LLMs are intelligent are like a cargo cult, or polytheists who believe their idols are gods. They're anthropomorphizing processes they don't understand. How can a probabilistic next-token word predictor be intelligent? Actual mammal brains are far more complex than that.
An LLM has never had any analogous physical experiences so it doesn't know what either is. In fact a text model has never seen a car.
You don't need experience, you need some basic concepts.
And it is precisely because higher mathematics is so far abstracted away from physical experience is the reason why many people struggle with it.
Besides, many great mathematicians have explained their knowledge as an "experience" even "spiritual experience". Not all experiences are physical. Ramanujan described his experience of mathematics as lights and sounds and patterns, and this is common among geniuses.
> you need some basic concepts
Agreed. All basic concepts are an experience.
There are a ton of things you can learn without experiencing it yourself and without analogues. For example, car crashes are bad to be in. Even though I have not personally experienced one, nor have I experienced some analogue of it.
https://plato.stanford.edu/entries/qualia-knowledge/
https://courses.physics.illinois.edu/phys419/sp2021/Jackson1...
It’s frightening to think the people are already equating LLMs and humans. LLMs use text created by humans to create more text that resembles text written by humans. There is no comparison between the two. Every human is capable of acquiring new knowledge and experiences. An LLM can only acquire what has already been acquired by some human.
Similar tech in the latest Gemini I think.
For the nth+1: well wired humans have built their systems trained on double checking and distrusting their own ideas.
-- Natural intelligences learn - after reflection, the mind is updated (hopefully upgraded): we need dynamic Neural Networks in the systems.
-- Ideas honing, and the perfectioning of the World Model and cognitive systems behind them, should be iterative and continuous...
But for most practical use cases we don't want the persistent memory. We already have opportunistic fact extraction like chatgpt profile building if we want that. We also have graph builders.
This mostly covers the private usage. Public version was attempted and just ends up like this: https://en.wikipedia.org/wiki/Tay_(chatbot)
If you like the metaphors (which this article seems to come close to), you can have a sort of "unconscious" and a "conscious" - where the "unconscious" may be somehow similar to Kant's "trascendental" (interpreted as the mind's Operating System before experience).
> Tay
Well, if the bot becomes "opinionated", it's probably just immature - just like a human. You will tell it to ponder well texts like Adam Rutherford's How to Argue With a Racist...
But surely we want an Intelligence that forms its own opinions: we want to understand and manage reality, so we should hope that a super-Chomsky and a super-Ferguson will at some point converge and at some point put their "identity"-related positions into perspective.
-- same
Whereas (for clarity) everytime one thinks he learns something and hones some skill.
Were it so, anyway, it still does not seem like the process of "pondering" (that seems to be more like "absorbing information").
This is not a matter of belief, it's a matter of observable fact.
Humans reason. LLM predict.
> Humans reason
Are you sure? They reason in a way to match others predictions, right? The problem of humans transporting information from one individual to another remains. All reason can effectively only be prediction, as it requires two persons at minimum. And there is a lot of noise to filter.
> LLM predict.
Well, there is a human that needs to reason with what the LLM has predicted. So from that perspective, it should suffice already for the moment.
And are LLM's unable to reason?
«Humans [can] reason. LLM [in a way just] predict.»
"Best interpretation."
So when they see LLMs say "I don't know", their reaction is "it doesn't know what it is talking about", not "wow LLMs actually can tell what it knows!"
Conversely, when they observe a human confidently says something wrong, their reaction is "what a cocky guy" but not "oh so humans are not better than LLMs in this regard."
Because they already believe humans are different and superior, whatever they observe doesn't affect their opinion.