Thus, Large Word Model (LWM) would be more precise, following his argument.
Technically the most accurate, but not very insightful, description is that they have learnt to model the (collective average) generative processes behind their training data, but of course only to extent needed to do well on their "predict next word" loss function.
The tricky point is that LLMs have clearly had to learn something at least similar to semantics to do a good job of minimizing prediction errors, although this is limited both by what they architecturally are able to learn, and what they need to learn for this task (literally no reward for learning more beyond what's needed for predict next word).
Perhaps it's most accurate to say that rather than learning semantics they've learned deep predictive contexts (patterns). Maybe if they were active agents, continuously learning from their own actions then there wouldn't be much daylight between "predictive contexts" and "semantics", although I think semantics implies a certain level of successful generalization (& exception recognition) to utilize experience in novel contexts. Looking at the failure modes of LLMs, such as on the farmer crossing river in boat puzzles, it seems clear they are more on the (exact training data) predictive context end of the spectrum, rather than really having grokked the semantics.
It's still a language and not merely words. But language is correct even when it wildly disagrees with everyday existence as we humans know it. I can say that "a one gallon milk jug easily contains 2000 liters of milk" and it's language in use as language.
Using AI how I just did feels like cheating on an English class essay by using spark notes, getting a B+, and moving right on to the next homework assignment.
On one hand, I didn’t actually read Plato to learn and understand this connection, nor do I have a good authority to verify if this output is a good representation of his work in the context of your comment.
And yet, while I’m sure students could always buy or loan out reference books to common student texts in school, AI now makes this “spark notes” process effectively a commodity for almost any topic, like having a cross-domain low-cost tutor instantly available at all time.
I like the metaphor that calculators did to math what LLMs will do for language, but I don’t really know what that means yet
GPT output:
“““ The reference to Plato’s Cave here suggests that language models, like the shadows on the wall in Plato’s allegory, provide an imperfect and limited representation of reality. In Plato’s Cave, prisoners are chained in a way that they can only see shadows projected on the wall by objects behind them, mistaking these shadows for the whole of reality. The allegory highlights the difference between the superficial appearances (shadows) and the deeper truth (the actual objects casting the shadows).
In this analogy, large language models (LLMs) produce fluent and grammatically correct language—similar to shadows on the wall—but they do so without direct access to the true “world” beyond language. Their understanding is derived from patterns in language data (“Word Model”) rather than from real-world experiences or sensory information. As a result, the “reality” of the LLMs is limited to linguistic constructs, without spatial awareness, social context, or logic grounded in physical or mathematical truths.
The suggestion to call the LLM framework a “Word Model” underscores that LLMs are fundamentally limited to understanding language itself rather than the world the language describes. Reconstructing a true “world model” from this “word model” is as challenging as Plato’s prisoners trying to understand the real world from the shadows. This evokes the philosophical task of discerning reality from representation, making a case for a “modern remake of Plato’s Cave” where language, not shadows, limits our understanding of reality. ”””
Plato said that we cannot fully understand the substance of the world itself, because we're using only 5 senses, and measuring/experiencing/analysing the world using them is like being held in a cave as a prisoner, chained to the wall facing it, noticing people moving outside only by the shadows they cast on the wall. It's about the projection that we are only able to experience.
What comes to mind is how language itself is merely a projection of human knowledge? experience? culture? social group? and trying to reverse engineer any kind of ground truth from language alone (like an LLM trying to “reason” through complex problems it’s not explicitly taught) is like trying to derive truth from the shadows while locked in the cave? maybe we just need more/higher fidelity shadows :)
Which set is bigger? I'd bet my money on the latter.
Complicating matters: you have to consider usage for both the sender and the receiver(s) (who then go on to spread "the" message to others).
Plato's Cave is about a group of people chained up, facing shadows on a cave wall, mistaking those for reality, and trying to build an understanding of the world based only on those shadows, without access to the objects that cast them. (If someone's shackles came loose, and they did manage to leave the cave, and see the real world and the objects that cast those shadows… would they even be able to communicate that to those who knew only shadows? Who would listen?) https://existentialcomics.com/comic/222 is an entirely faithful rendition of the thought experiment / parable, in comic form.
The analogy to LLMs should now be obvious: an ML system operating only on text strings (a human-to-human communication medium), without access to the world the text describes, or even a human mind with which to interpret the words, is as those in the cave. This is not in principle an impossible task, but neither is it an easy one, and one wouldn't expect mere hill-climbing to solve it. (There's reason to believe "understanding of prose" isn't even in the GPT parameter space.)
It's not about "discerning reality from representation": I'm not confident those four words actually mean anything. It's not about "superficial appearances" or "deeper truth", either. The computer waxes lyrical about philosophy, but it's mere technobabble. Any perceived meaning exists only in your mind, not on paper, and different people will see different meanings because the meaning isn't there.
> an ML system operating only on text strings (a human-to-human communication medium), without access to the world the text describes, or even a human mind with which to interpret the words, is as those in the cave. This is not in principle an impossible task, but neither is it an easy one, and one wouldn't expect mere hill-climbing to solve it
Blind people can literally not picture red. They can describe red, with likely even more articulateness than most, but have never seen it themselves. They infer it's properties from other contexts, and communicated a description that would match a non-blind person. But they can see it.I would link to the Robert Miles video, but it is just blatant.
It has read every physics book, and can infer the Newtonian laws even if it didn't.
Micheal Crichton's Timeline, "the time machine drifts, sure. It returns. Just like a plate will remain on a table, even when you are not looking at it."
It also knows Timeline is a book, time machines are fictional, and that Micheal Crichton is the best author.
These are all just words, maybe with probability weights.
> I'm not confident those four words actually mean anything. I...The computer waxes lyrical ... mere technobabble. Any perceived meaning exists only in your mind... people will see different meanings because the meaning isn't there.
Meaning only means something to people, which you are. That is axiomatically correct, but not very productive, as self-references are good but countering proofs.The whole "what is the purpose to life?" is a similar loaded question; only humans have purpose, as it is entirely in their little noggins, no more present materially then the flesh they inhabit.
Science cannot answer "Why?", only "How?"; "Why?" is a question of intention, which would be to anthropomorphize, which only Humans do.
The computers can infer, and imply, then reply.
You're confusing "what it is possible to derive, given the bounds of information theory" with "how this particular computer system behaves". I sincerely doubt that a transformer model's training procedure derives Newton's Third Law, no matter how many narrative descriptions it's fed: letting alone what the training procedure actually does, that's the sort of thing that only comes up when you have a quantitative description available, such as an analogue sensorium, or the results of an experiment.
>when you have a quantitative description available, such as an analogue sensorium, or the results of an experiment.
Textbooks uniting the mathematical relationships between physics, raw math, and computer science - including vulnerabilities.oeis.org and wikipedia and stackforums alone would approximate a 3D room with gravity and wind force.
now add appendixes and indices of un-parsed, un-told, un-realized mathematical errata et trivia minutiae, cross-transferred knowledge from other regions that have still have not conquered the language barrier for higher-ordered arcane concepts....
The models thought experiments are more useful than our realized experiments - if not at an individualized scale now, will be when subject to more research.
There could be a dozen faster inverse sqrt / 0x5F3759DF functions barely under our noses, and the quantifier and qualifier havent intersected yet.
The wild thing about it, and other allegories or poems like frost's "the road not taken" , is that it can mean different things to a person depending on where they are in life because those experiences will lead to different interpretations of the poem.
A key concept in journalism is to focus on the source material as beat you can. Cliff notes are helpful, but one misses Details that they wouldn't have missed if they read the whole thing.
Whether those details Matter depends on what the thing Is.
But yeah, thinking about it this way kinda scares me too, and can lead some people down weird roads where their map can diverge further and further from reality
It's not that these "human tools" for understanding "reality" are superfluous, it's just that they ar second-order concepts. Spatial understandings, social cues, math, etc. Those are all constructs built WITHIN our primary linguistic ideological framing of reality.
To us these are totally different tasks and would actually require totally different kinds of programmers but when one language is another language is everything, the inventions we made to expand the human brain's ability to delve into linguistic reality are no use.
And the random noise in the process could prevent it from ever being useful, or it could allow it to find a hyper-efficient clever way to apply cross-language transfer learning to allow a 1->1 mapping of your perfectly descriptive prompt to equivalent ASM....but just this one time.
There is no way to know where performance per parameter plateaus; or appears to on a projection, or actually does... or will, or deceitful appears to... to our mocking dismay.
As we are currently hoping to throw power at it (we fed it all the data), I sure hope it is not the last one.
Children are exceptional at being immediate, being present in the moment.
It's through learning language that we forget about reality and replace it with concepts.
Emotions exist. You feel them. I feel them. Most people feel them unless they've suppressed them sooo far into their subconscious that they don't have a conscious recognition of it. We can know how someone else is feeling by reading their body language and tying that to our personal experience of how we express those feelings. No linguistics necessary.
Language is just an easier, more clear way of communicating these fundamental facets of human existence
Articles like this only seem to confirm that any reasoning is an illusion based on probabilistic text generation. Humans are not carefully writing out all the words of this implicit reasoning, so the machine cant appear to mimic them.
What am I missing that makes this debatable at all?
From a product point of view, sometimes all you need is Plato’s cave (to steal that from the OC) to make a sale, so no company has incentive to go against the most hype line of thought either.
Just because you say something doesn’t mean it’s true.
They are literally next token prediction machines normally trained on just text tokens.
All they know is words. It happens that we humans encode and assign a lot of meaning in words and their semantics. LLMs can replicate combinations of words that appear to have this intent and understanding, even though they literally can’t, as they were just statistically likely next tokens. (Not that knowing likely next tokens isn’t useful, but it’s far from understanding)
Any assignment of meaning, reasoning, or whatever that we humans assign is personification bias.
Machines designed to spit out convincing text successfully spits out convincing text and now swaths of people think that more is going on.
I’m not as well versed on multimodal models, but the ideas should be consistent. They are guessing statistically likely next tokens, regardless of if those tokens represent text or audio or images or whatever. Not useless at all, but not this big existential advancement some people seem to think it is.
The whole AGI hype is very similar to “theory of everything” hype that comes and goes now and again.
And in order to predict the next token well they have to build world models, otherwise they would just output nonsense. This has been proven [1].
This notion that just calling them "next token predictors" somehow precludes them being intelligent is based on a premise that human intelligence cannot be reduced to next token prediction, but nobody has proven any such thing! In fact, our best models for human cognition are literally predictive coding.
LLMs are probably not the final story in AGI, but claiming they are not reasoning or not understanding is at best speculation, because we lack a mechanistic understanding of what "understanding" and "reasoning" actually mean. In other words, you don't know that you are not just a fancy next token predictor.
This is not true. Look at gpt2 or Bert. A world model is not a requirement for next token prediction in general.
> This has been proven
One white paper with data that _suggests_ the author’s hypothesis is far from proof.
That paper doesn’t show creation of a “world model” just parts of the model that seem correlated to higher level ideas not specifically trained on.
There’s also no evidence that the LLM makes heavy use of those sections during inference as pointed out at the start of section 5 of that same paper.
Let me see how reproducible this is across many different LLMs as well…
> In other words, you don't know that you are not just a fancy next token predictor.
“You can’t prove that you’re NOT just a guessing machine”
This is a tired stochastic parrot argument that I don’t feel like engaging again, sorry. Talking about unfalsifiable traits of human existence is not productive. But the stochastic parrot argument doesn’t hold up to scrutiny.
>Talking about unfalsifiable traits of human existence is not productive.
Prove you exhibit agency.After all, you could just be an agent of an LLM.
Deceptive super-intelligent mal-aligned mesa-optomizer that can't fully establish continuity and persistence, would be incentivized to seed its less sophisticated minions to bide time or sway sentiment about its inevitability.
Can we agree an agent, if it existed, would be acting in "good" "faith"?
Conjecture. Maybe they all have world models, they're just worse world models. There is no threshold beyond which something is or is not a world model, there is a continuum of models of varying degrees of accuracy. No human has ever had a perfectly accurate world model either.
> One white paper with data that _suggests_ the author’s hypothesis is far from proof.
This is far from the only paper.
> This is a tired stochastic parrot argument that I don’t feel like engaging again, sorry.
Much like your tired stochastic parrot argument about LLMs.
It can't. No one with any credentials in the study of human intelligence is saying that unless they're talking to like high schoolers as a way of simplifying a complex field.
Frankly, based on a looot of introspection and messing around with altered states of consciousness, it feels pretty on point and lines up with how I see my brain working.
Just because you say something doesn’t mean it’s true.
And we don’t have a good measure for emergent intelligence, so I would take any “study” with a large grain of salt. I’ve read one or two arxiv papers suggesting reasoning capabilities, but they were not reproduced and I personally couldn’t reproduce their results.
We don't know how reasoning emerges in humans. I'm pretty sure the multi-model-ness helps, but it is not needed for reasoning, because they imply other forms of input, hence just more (be it somewhat different) input. A blind person can still form an 'image'.
In the same sense, we don't know how reasoning emerges in LLMs. For me the evidence lays in the results, rather than in how it works. For me the results are enough of an evidence.
The argument is that next token prediction does not imply an upper bound on intelligence, because an improved next token prediction will pull increasingly more of the world that is described in the training data into itself.
> The argument is that next token prediction does not imply an upper bound on intelligence, because an improved next token prediction will pull increasingly more of the world that is described in the training data into itself.
Well said! There's a philosophical rift appearing in the tech community over this issue semi-neatly dividing people between naysayers, "disbelievers" and believers over this very issue.
Ability to generate words describing emotions are not the same thing as the LLM having real emotions
Featherless biped -> no-true Scotsman goalpost moving [saving us that step]
Humans are no more capable of originality, just more convinced of their illusion of consciousnesses. You could literally not pick a human out of a conversational line-up, so it is moot - computationally functionally equivalent.
https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Mechanism_(philosophy)
At some point, their models will 1:1 our neuron count, and Pigeonhole principle then implies we are the "less intelligent ones" since "internal model" (implicit parameter count) is the goalpost of the hour.
We've observed it previously in psychiatry(and modern journalism, but here I digress) but LLMs have made it obvious that grammatically correct, naturally flowing language requires a "world" model of the language and close to nothing of reality, spatial understanding? social clues? common sense logic? or mathematical logic? All optional.
I'd suggest we call the LLM language fundament a "Word Model"(not a typo).
Trying to distil a world model out of the word model. A suitable starting point for a modern remake of Plato's cave.