Re:Chomsky and the Two Cultures of Statistical Learning
by robotresearcher on Friday November 06, 2026 @11:56AM
(#50394933)
> Who'd have thought that mere statistics could do that?
Statistical models can approximate arbitrary functions, so, really anyone who knows what that means, and who believes intelligent behavior is a physical process.
Re:Chomsky and the Two Cultures of Statistical Learning
by theptip on Friday November 06, 2026 @06:26PM
(#50399903)
“Mere statistics” is heavily underselling what the transformer architecture does. Unless you think “mere ion channels” is a sensible way to think about the human brain.
Re:Chomsky and the Two Cultures of Statistical Learning
by hocusPokeUs on Friday November 06, 2026 @09:24PM
(#50401049)
"Mere statistics" is how physical reality works; I don't mean the language, the syntax
I mean shifting influence of gravity relative to an object with enough mass type ratios are everywhere
The language of math started as visual art to depict such ratios; in modern times this ground truth associations is frequently overlooked or never understood to exist in the first place without engagement
Differential diagnosis in medicine is exactly the same process; such processes and their successes are everywhere.
The issue we have in software is the outcome isn't as obvious as jabbing a blood draw needle in a patients eye; we spent a lot of time wasting time on illusory needs
There's pretty interesting (I think) research in how effectively it's possible to scale down LLMs. If you're interested, take a look at the BabyLM conferences:
https://babylm.github.io/
(I have no connection to them.) The work there demonstrates that gap between what we can do now with human-relevant amounts of training text, and what remains to be discovered. Quite a bit, it looks like.