LLMs cheat at generating text because they do so via a model of the statistical structure of text.
We're in the world, it is us who stipulate the meaning of words and the structure of text. And we stipulate new meanings to novel parts of the world daily.
What else is an 'iPhone' etc. ? There's nothing in `i P h o n e` which is at all like an iphone.
We have just stipulated this connection. The machine replays these stipulations to us -- it does not make them, as we do.
Consider two people - one, a Papau New Guinea tribesperson from a previously uncontacted tribe who is allowed to handle a powered-down iPhone, and told it is an "iPhone", but is otherwise ignorant of its behavior - the other, a cross-platform mobile software developer who has never actually held a physical iPhone, but is intimately familiar with its build systems, API, cultural context etc. Between the two of them, who better understands what an iPhone "is"?
You make a good point about inventing words to refer to new concepts. There's nothing theoretically stopping a language model from identifying some concept in its training data that we don't have a word for, inventing a word for it, and using it to give us a perspective we hadn't considered. It would be very useful if it did that! I suspect we don't tend to see that simply because it's a very rare occurrence in the text it was trained on.
A concept is a sensory-motor technique abstracted into a pattern of thought developed by an animal, in a spatio-temporal environment, for a purpose.
LLMs are just literally an ensemble of statistical distributions over text symbols. In generating text, they're just sampling from a compressed bank of all text ever digitised.
We aren't sampling from such a bank, we develop wholey non-linguistic concepts which describe the world, and it is these which language piggy-backs on.
The structure of symbols in a book has nothing to do with the structure of the world -- it is we who have stipulated their meaning: there's no meaning to `i`
Text is an extremely limited input stream, but an input stream nonetheless. We know that animal intelligence works well enough with any of a range of sensory streams, and different levels of emphasis on those streams - humans are somehow functional despite a lack of ultrasonic perception and primitive sense of smell.
And your definition of a concept is quite self-serving... I say that as a mathematician familiar with many concepts which don't map at all to sensory motor experiences.
Sensory-motor expression of concepts is primitive, yes, they become abstracted --- and yes the semantics of those abstractions can be abstract. I'm not talking semantics, i'm talking genesis.
How does one generate representations whose semantics are the structure of the world? Not via text token frequency, this much is obvious.
I dont think the thinnest sense of "2 + 2 = 4" being true is what a mathematician understands -- they understand, rather, the object 2, the map `+` and so on. That is, the proposition. And when they imagine a sphere of radius 4 containing a square of length 2, etc. -- I think there's a 'sensuous, mechanical, depth' that enables and permeates their thinking.
The intellect is formal only in the sense that, absent content, it has form. That content however is grown by animals at play in their environment.
Hi, since human linguistics is the sole repository of linguistic conceptualism, can you please show me which of the neurons is the "doggie" neuron, or the "doggie" cluster of neurons? I want to know which part of the brain represents the thing that goes wag-wag.
If you can't mechanically identify the exact locality of the mechanism within the system, it doesn't really exist, right? It's just a stochastic, probabilistic model, humans don't understand the wag-wag concept, they just have some neurons that are weighted to fire when other neurons give them certain input stimuli tokens, right?
This is the fundamental problem: you are conflating the glue language with the implementation language in humans too. Human concepts are a glue-language thing, it's an emergent property of the C-language structure of the neurons. But there is no "doggie" neuron in a human just like there is no "doggie" neuron in a neural net. We are just stochastic machines too, if you look at the C-lang level and not the glue-language level.
But then also consider the following: a human being from 2006, and an LLM that has absorbed an enormous corpus of words about iPhones that is also granted access to a capacitive-touchscreen friendly robot arm and continuous feed digital camera (and since I'm feeling generous, also a lot of words about the history and architecture of robot arms and computer vision). There is no doubt the LLM will completely blow the human out of the water if asked trivia questions about the iPhone and its ecosystem.
But my money's on the 2006 human doing a lot better at switching it on and using the Tinder app...
It’s especially hard to parse a dark sweeping condemnation based on…people are investing in it? It doesn’t have the right to assign names to things? Idk what the argument is.
My most charitable interpretation is “it cant reason abour anything unless we already said it” which is obviously false.
The point is that they're not those things. Yes, language models can produce solutions to language tests that a 14 year old could also produce solutions for, but a calculator can do the same thing in the dimension of math - that doesn't make a calculator a 14 year old.
I removed the reference, in retrospect, it’s unnecessary. No need to indicate the strong performance, we’re all aware.
The point is not so much that we already said it, more that the patterns it encodes and surfaces when prompted are patterns in the written corpus, not of the underlying reality (which it has never experienced). Much like a list of all the addresses in the US (or wherever) will tell you very little about the actual geography of the place.
You've never experienced the "underlying reality" either.
No we don't. Humans don't experience or perceive reality. We perceive a nice modification of it and that's after excluding all sense data points we simply aren't capable of perceiving at all.
Your brain is constantly shifting and fabricating sense data based on internal predictions and that form the basis of what you call reality. You are not learning from the structure of the world. You are learning from a simplified model of it that is fabricated at parts.
This just comes down to giving transformers more modalities, not just text tokens.
There is nothing about “2” that conveys any “twoness”, this is true of all symbols.
The token “the text ‘iphone’” and the token “visual/tactile/etc data of iphone observation” are highly correlated. That is what you learn. I don’t know if you call that stipulation, maybe, but an LLM correlates too in its training phase. I don’t see the fundamental difference, only a lot of optimizing and architectural improvements to be made.
Edit: and when I say “a lot”, I mean astronomical amounts of it. Human minds are pretty well tuned to this job, it’ll take some effort to come close.
> Humans learn from the structure of the world -- not the structure of language.
In short, there's no evidence that being in the world makes our knowledge any different from an LLM. The main advantages we have at the moment are sensory learnings (LLMs are not good at comparing smells and flavors) and the ability to continuously train our brains.
It almost doesnt matter what your theory of language is --- any even plausible account will radically depart from the above statistical model. There isn't any theory of language which supposes it's an induction across text tokens.
The problem in this whole discussion is that we know what these statistical models are (models of association in text tokens) -- yet people completely ignore this in favour of saying "it works!".
Well "it works" is NOT an explanatory condition, indeed, it's a terrible one. If you took photographs of the night sky for long enough, you'd predict where all the stars are --- these photos do not employ a theory of gravity to achive these.
LLMs are just photographs of books.
There's a really egregious pseudoscience here that the hype-cycle completely suppresses: we know the statistical form of all ML models. We know that via this mechanism arbitrarily accurate predictions, given arbitrarily relevant data, can be made. We know that nothing in this mechanism is explanatory.
This is trivial. If you video tape everything and play it back you'll predict everything. Photographing things does not impart those photographs the properties of those things -- those serve as a limited assocative model.
I don’t have a reference handy now (someone can probably do better) but I believe one way to see this is via the hearing impaired or hearing and sight impaired
Humanoids went millions of years literally learning to navigate 3D space and sense “enough heat, food, water” etc
Nomadic tribes had built shared resource depots millennia before language.
I can see the color gradients of the trees and feel muscles relax without words.
Human language beyond some utilitarian labels just instills mind viruses that bloom into delusions of grandeur.
90% of human communication is unspoken. Neuroscience shows our brains sync behavior patterns with touch and just being in a room.
Reality is full of unseen state change every moment that we have no colloquial language for; human language is hardly the source of truth and the “North star” of human society in reality.
This is as scientific as the idea of humans just using 10% of our brains.
There’s just as little science language motivates me to work. Most of the language society relies on is hallucinations; fiat currency, nation states, constructs like “Senate” and Congress, corporatism, brands, copy-paste of historical terminology, not evidence they’re immutable features of reality.
What we recite has nothing to do with what we are. I find the appeals to non-existent political truisms primate gibberish.
It seems perfectly clear to me many facts of society are just memorized and recited prompt hacks. Language is the goto tool for propagandists, to obscure sensory connection to reality.
There is over 100 years of propaganda research available, too much for me to sort through, but scientific measure of such is not new; new to anyone unaware of it but not to humanity.
Just because it intuitively makes sense doesn’t make it correct or the 90% figure accurate.
> Steven Pinker, author of The Language Instinct, claims that "The Nicaraguan case is absolutely unique in history ... We've been able to see how it is that children—not adults—generate language, and we have been able to record it happening in great scientific detail. And it's the only time that we've actually seen a language being created out of thin air."
We can also look at creole languages. When distinct linguistic groups come into contact without a common language they will, in some cases, develop "pidgin" languages to communicate. These languages are created by the adults who are already fluent speakers of their native language. What is interesting is what happens with children born into such communities. They grow up hearing the pidgin language, but they do not learn the pidgin. Instead, the language they learn is a creole. It is based on the pidgin, but more complete and consistent with the way human language works in general. In effect, like all children, they are relying on their innate knowledge of language. The pidgin fills in the language specific parameters, but when the pidgin contradicts an aspect of innate language the children (for the most part), just don't aquire that portion of the language.
Fine.
We know that languages did not exist at some point. We made them exist.
Whether that happens in 1 generation or 10,000 generations is completely unknown.
I like to think of the brain as the general model we've trained with evolution, and the person's experiences as the specialization.
LLMs are not doing this.
This is a complete tautology in terms, and any dispute must therefore be over the ontology in which it is expressed. To elaborate how something came to be is not required to express that something is.
Evolution optimized for language learning abilities before the full capabilities came. Evolution is not thin air.
No...they aren't. Humans aren't learning from thin air by any stretch of the imagination.