but from all models, it's the closest, "all models are wrong, but some are useful"
We considered the hypothesis that humans are a general intelligence that figures language out by exposure. The evidence has consistently come back that that is not how humans actually work. A model where humans are hardwired with a lot of knowledge about language, then fill in the gaps based on exposure still appears to be far more accurate.
The only thing LLMs show is that it is possible to learn language de-novo.
I agree, but would add, "But not under conditions that humans learn language."
Regarding "hard-wired"; it's not if you consider that "hard-wiring" of an individual is the result of millions of years of evolution.
Care to back this up? What evidence do we have to suggest that back propagation and self attention are mechanistically similar to human cognition? Similarity of the outputs says nothing about similarity of the machinery underneath. Turboprop engines tell us nothing about how birds fly - doesn't matter that they successfully transport millions of people around the world.
I would say that when it comes to higher level cognition, logic engines are more human like even if they only have a tiny fraction of the practical utility of LLMs.
That is not true.
Turboprop engines and flying birds are different things, but there is a set of common properties.
A blade of a screw propeller functions exactly in the same way as a flapping wing of a bird, except that the wing cannot rotate without limits around the body of the bird, so it must stop, change angles and revert the motion, while the blade can rotate continuously around the axis, functioning on one side like a falling wing and on the other side like a raising wing.
"Nothing" may be a little too strong -- physics is physics, after all -- but (small) birds and airplanes have Reynolds numbers about 1000x different and different underlying material science.
Airplanes are engineered (with a very small number of exceptions) to be stable during flight, whereas bird flight is inherently unstable, requiring a lot of specialized cognition so birds can fly.
I would think a lot of cognitive ornithologists (who I really hope exist!) would be interested in finding out about this.
At this point, I'm at the complete end of my knowledge of bird flight!
This is completely missing the forest for the trees. The "except" in your argument is doing an incredible amount of work. I think it's absurd to claim that the rotating blades of a turboprop are analagous to a bird's wings when planes l have separate structures quite literally called "wings" because they are directly inspired by bird wings. And I think the absurdity of that argument actually serves to prove my point.
And I'd be hard pressed to identify many cognitive characteristics of LLMs during execution that resemble human thinking, aside from those that are readily parrotable.
As a model of human cognition, LLMs deep suck.
in college, i was somewhat involved in cogsci wrt semantic mapping of linguistic concepts based on human surveys (this was well before LLM/AI-word2vec), and it surprises me today how the mathematical properties of relatedness/distance seem now to drive machine intelligence, something which we did not consider at the time at all.
maybe i'm getting in my metaphysical sentimental stage, a turn a lot of adults make after middle age, but i wouldn't fully dismiss the fact that there may be an equivalence.
There's pretty interesting (I think) research in how effectively it's possible to scale down LLMs. If you're interested, take a look at the BabyLM conferences:
(I have no connection to them.) The work there demonstrates that gap between what we can do now with human-relevant amounts of training text, and what remains to be discovered. Quite a bit, it looks like.
Statistical models can approximate arbitrary functions, so, really anyone who knows what that means, and who believes intelligent behavior is a physical process.
I mean shifting influence of gravity relative to an object with enough mass type ratios are everywhere
The language of math started as visual art to depict such ratios; in modern times this ground truth associations is frequently overlooked or never understood to exist in the first place without engagement
Differential diagnosis in medicine is exactly the same process; such processes and their successes are everywhere.
The issue we have in software is the outcome isn't as obvious as jabbing a blood draw needle in a patients eye; we spent a lot of time wasting time on illusory needs
Can you use a human language well enough and it not sound intelligent? And no, I don't mean just syntactically correct because semantics is a part of human languages too. I think the astounding thing is that they managed to model a language this well. It's like trying to learn English without a functional understanding of the word "Please".
My take is that a lot of the intelligence we natively ascribe to humans exist in human languages and just like the nutrients in your food, by harnessing a language that possesses latent intelligence, you too can behave like an intelligent person. And how did the nutrients of intellect got in there in the first place? Just like languages evolve nowadays: via contributions from speakers over time. Not unlike soil fertilisation.
For example, if I compose your post as :
--
"Good prose embodies intellect gradually evolved by myriad humans over time, so it wows me to see LLMs create it, though I soon realised that the intelligence is not being created by the LLM but rather is already present in the human-evolved input datasets - the prose itself! - which they statistically arrange."
--
..the particulars of the language are entirely different (and omit some of your chosen nuance) but most readers would receive the same "core truth" from it, the same measure of beneficial information.
Which highlights the great fluidity of language; that is another of the key underlying mechanisms allowing LLMs to flourish.
Regardless, the fluidity of language is self-evident to an extent : it is partly embodied in the entire concept we call figures-of-speech (a linguistic phrasing can be literal, or sarcastic, or described by metaphor, ...). Partly in the common reality that our dictionaries list multiple possible definitions for many words. Partly in the fact that (many) synonyms exist. Partly in sheer human individuality.
It's funny, I used to think it was complete and utter horseshit when literature professors would say that the meaning of a text is created by the reader, not the author.
Since LLMs have become commonplace, I have completely reversed my opinion on that.
That you see recognizable intelligence in the output of stochastic generation is because you are imposing meaning on it. It is why "claudish" is so hard to read--you are trying to impose meaning on a sequence of words that was stochastically generated, and it takes mental effort to do that.
> you see recognizable intelligence in the output of stochastic generation is because you are imposing meaning on it
If Claude can write formal mathematical proofs (that a machine can check), or it can write software that works, where's the head of the reader that imposes the meaning?
> a sequence of words that was stochastically generated
As was your comment (and mine, and the others on this page and elsewhere).
It most certainly does not. It's obvious to anyone who critically examines LLMs, without letting the hype override sense, that there's zero intelligence there.
"This year, we have merged the Multimodal and Interaction tracks into the Strict and Strict-small tracks. This is for two primary reasons: Firstly, there were insufficient submissions to warrant standalone tracks. Secondly, none of the submissions in the Interaction track outperformed models submitted to the Strict track, and *none of the submissions from Multimodal outperformed our baselines*. We will still allow participants to use multimodal data or interactive training procedures if they choose, but such data or learning procedures must conform to the rules of the Strict or Strict-small tracks.
Emphasis mine.
The multimodal baselines are _lower_ than the text only baselines. See Table 3. In fact, 10M of text outperforms 100M of text + images.
That is, you can apply for grants and write papers in the transformational grammar paradigm endlessly. You can pose and answer questions that make sense within in the system.
Chomskyism was disappointing in two respects: (1) was that it was an inadequate basis for machine understanding of text, (2) was that structuralists were hoping everything in culture could be understood in terms of "... having the structure of language" but syntactic structures weren't all adequate for that task.
(1) and (2) both pointed to Chomskyism being hermetic and not something really useful or inspiring to other fields and maybe just a little less "true".
Yes this is one of my critiques of the academic system (I have finished a PhD and will hopefully never work at a university ever again). Many papers in the introduction start with explaining how the study builds on plenty of previous research, but that alone doesn’t prove that something is a good direction to investigate. If academics were completely honest, many papers would write “We researched topic X because we got funding for it. But since we actually don’t care about X, we just wrote a proposal about our favorite niche topic that only one of the co-authors cares about, namely topic Y, and wrote it such that it sounds like X. Next we show that Y is actually very cool in the hope to convince others that it is cool plus to convince others of the conclusion that our most senior professor already found during the PhD which was 30 years ago. Apart from the professor, we don’t believe in the main argument of the paper but since he got the grant we’ll just go with it in the hope to one day be able to push our own ideas forward”.
Also, I'll mention that there is a lot of empirical knowledge about natural language syntax in generative grammar, even if the details aren't quite right about their theoretical underpinnings. And a lot of them were "low-hanging fruit" waiting for someone to point out how to find them and characterize them. That Chomsky did that is the best argument for his intellectual value - permitting all that "normal science" to happen.
Science doesn't guarantee technological applications, whether or not that science is correct.
Finally, just saying: use of a word like "Chomskyism" makes you sound like one of those creationists who call evolution "Darwinism".
This is wholly fallacious ... you may disagree with what you interpret as the implications of attaching "ism" to Chomsky, but in no way does that equate it to attaching "ism" to Darwin -- each of these attachments and their validity are independent.
BTW, even what you see as pejorative in such an attachment is a cultural interpretation ... e.g., it is far more common among British evolutionary biologists than American to view "Darwinism" neutrally -- Dawkins for one uses the term and describes himself as a Darwinist. See, e.g., https://www.edge.org/conversation/richard_dawkins-chapter-3-...
this distinction is superficial. Scientists use less precise models all the time. Like, you don't solve the schrodinger equation even when you're a scientist doing organic chemistry. You reach for lewis dot models. Science as a discipline is about building better models of the world, not uncovering fundamental truth.
I don’t think Norvig is confused. He explicitly addresses this point right at the top of the article.
His argument is that successfully engineering a system using a statistical method provides some bits of evidence for the hypothesis that the system itself is statistical.
Which is obviously true, so long as we take care to weigh evidence properly.
> there's absolutely no reason to think that tells you anything scientific about the phenomenon you're modeling
No, this is not how evidence works. If you can accurately model a system with method X and you cannot come up with a model that works with method Y, you should think it’s more likely (but not certain) that X is the actual physical model.
And indeed modern cognitive science (see Predictive Processing) also has a bunch of evidence for the brain as a hierarchical predictor, rather than a rule engine.
To your points - planes are not model organisms of birds, but they do shed insight on the bounds that birds must obey, eg thrust-to-weight etc. And the sample efficiency point really needs to die. The human brain is born with an extremely complex prior baked in. An LLM begins pretraining with almost nothing (just a very simple architecture and a few simple algorithms like attention), and has to learn the basic architecture of cognition from scratch. A better baseline would be how many samples were processed by every animal in our genetic ancestry that had a brain, since brains existed.
Could you expand on this point?
I was also going to suggest coin flips, but it’s not quite the same… I think any chaotic system is generally best modeled that way, and if you cared about the effect the system has on other things, it’s unlikely you can do better since measuring the initial state will not be feasible.
Math is not physics and physics is not math. Writing a Superman story does not mean the language and pictures represent physics
That's the difference being highlighted here; stats are not evidence of gravity and electromagnetism; physical phenomena when materials influenced by those forces are relative to local gravity, electromagnetic fields are our evidence
Stats of 1960s demographics did not capture immutable physical phenomena because entropy-> generational churn -> those people are now dead
The languages do not matter; they're just to communicate some ground truth.
Shackling society to the past through careful statistical methods relative to finance and politics to avoid societal change is not a scientific conclusion; there's no reason to use 30-40 years ago as the center of our social universe
The concern with stats consumption is we walk away from a result as if its an immutable forever conclusion as we ignore that gravity and electromagnetism, etc have not stopped changing ground truth; past truths that no longer matter to our next choices
You can write many imagined stories, but one of them will match observed reality. It won't be about an airborne male model who wears his underpants on the outside. But you can write a story that describes reality. And you can describe observable phenomena with statistics. Of course, you can describe plenty of pseudoscientific drivel with statistics too.
Norvig confuses engineering for science. If an engineer wants something that works (and guarantees that it will work) then you use the most effective tool available to achieve that goal. No problem there. But there's absolutely no reason to think that tells you anything scientific about the phenomenon you're modeling. For example, aeronautical engineering says very little about how birds fly. And - currently relevant - LLMs learn to use and understand language, but they use millions of times more training data, and millions of times more computing power, than a child does. So an LLM may be a great technology, but it's probably not a good model for how humans learn and use language.
Chomsky OTOH doesn't see any use for statistics in science, at least not in linguistics. Fortunately for the field of linguistics, Chomsky's influence in linguistics is waning and linguists are beginning to use statistical tools to do science.