Re:How LLMs Work
by otabdeveloper4 on Saturday June 06, 2026 @10:15AM
(#48425343)
> to then make your model bigger, which then makes it smarter
There's diminishing returns and at some point making a model bigger makes it dumber.
Maybe due to lack of data and dimensions other than words.
Re:How LLMs Work
by TheHalfDeafChef on Saturday June 06, 2026 @09:42AM
(#48425044)
Not really “smarter” though? It’s just a big probability engine.
(Not trying to flame bait or anything. I just wouldn’t call LLM as exhibiting intelligence. It is great at making connections based on probability but doesn’t have a semantic understanding of what it is doing)
Re:How LLMs Work
by stevenhuang on Sunday June 07, 2026 @01:29PM
(#48436928)
You do realize modern neuroscience considers the human brain as "just" a probability engine and that intelligence may well be the ability for an organism to predict well.
> doesn’t have a semantic understanding of what it is doing
I hope you realize this is an area of open, active research.
Didn't neuroscience some big scandals about bad statistics and overstating their findings (in addition to normal issues like replication)? Look up at least the "dead salmon study" (hint: it's related to fMRI, and you can probably guess its conclusions from its nickname).
The "Voodoo Correlations" and "Cluster Failure" papers are also a bit eye-opening.
In general we (humans) need to be humble about the limitations of our knowledge about how we function, it's an insanely complicated problem.
> In general we (humans) need to be humble about the limitations of our knowledge about how we function, it's an insanely complicated problem.
We do.
Which is why we shouldn't be assuming we're more than just probability engines, or be assuming we have more consciousness than a neural network.
If you can make the existing model faster, you can then save your inference budget to then make your model bigger, which then makes it smarter.
A lot of how smart the models can be comes down to budget. If you can make your existing thing cheaper, you can instead make it bigger for the same price.