News for hackers. Stuff that matters.
Re:Using LLMs to Understand Language Processing in the Human... by cs702 on Tuesday March 25, 2025 @03:31PM (#43474941)
I view this as compelling evidence that current models are more than "stochastic parrots," because as the OP shows, they are learning to model the world in ways that are similar (up to a linear transformation) to those exhibited by the human brain. The OP's findings, in short:

* A linear transformation of a speech encoder's embeddings closely aligns them with patterns of neural activity in the brain's speech areas in response to the same speech sample.

* A linear transformation of a language decoder's embeddings closely aligns them with patterns of neural activity in the brain's language areas in response to the same language sample.

Re:Using LLMs to Understand Language Processing in the Human... by hacker_homie on Tuesday March 25, 2025 @04:11PM (#43475376)
I view this as the language processing is similar but we’re not talking about thinking, just processing language.

I see this as maybe it’s not a statistical parrot, but it’s still only some kind of parrot, Maybe a sleep deprived one.

[deleted]
Re:Using LLMs to Understand Language Processing in the Human... by Terr_ on Tuesday March 25, 2025 @05:25PM (#43476156)
> the vast number of internal models of the world that are generated strictly from text input

Do you mean in the sense than an LLM can emit documents following grammar rules or pirate-talk?

For humans, text doesn't generate our models so much as as it triggers them.

Re:Using LLMs to Understand Language Processing in the Human... by og_kalu on Tuesday March 25, 2025 @11:39PM (#43478653)
There isn't really any clear delineation between 'thinking' and 'language processing' in the brain.
Re:Using LLMs to Understand Language Processing in the Human... by aithrowawaycomm on Wednesday March 26, 2025 @03:15AM (#43479623)
That's 100% false, dogs and pigeons can obviously think, and it is childish to suppose that their thoughts are a sequence of woofs or coos. Trying to make an AI that thinks like a human without being able to think like a chimpanzee gives you reasoning LLMs that can spit out proofs in algebraic topology, yet still struggle with out-of-distribution counting problems which frogs and fish can solve.
Re:Using LLMs to Understand Language Processing in the Human... by wruza on Wednesday March 26, 2025 @07:48AM (#43481152)
How human brains process thoughts is non-uniform across population. There’s imagery, written language, sound, speech, tactile, etc. Not everything that you think about is readily expressible in your language. There are definitely people with and without “internal screen”, and probably few more types with/without X, where X is a set of things we’ve never talked about, either assuming everyone has it or not realizing that it’s a non-mandatory part of how you think.
Re:Using LLMs to Understand Language Processing in the Human... by og_kalu on Wednesday March 26, 2025 @08:14AM (#43481327)
That's not really what I'm saying. What I'm saying is how does the brain look when you do both ? Is there a clear difference. There's no 'thinking mode' followed by a 'language processing mode'. Language processing is thinking as far the brain is concerned and there's no evidence that these are 2 cleanly separated processes whether you 'think' in words or not.
Re:Using LLMs to Understand Language Processing in the Human... by foobarqux on Wednesday March 26, 2025 @12:23PM (#43483911)
That's not true, the Moro experiments show they use different capacities as do similar experiments on people who have certain severe cognitive deficiencies that don't impact language processing (e.g. the subject "Chris")
Re:Using LLMs to Understand Language Processing in the Human... by og_kalu on Wednesday March 26, 2025 @01:38PM (#43484663)
My argument is that “thinking” and “language processing” are not two sequential or clearly separated modes in the brain but deeply intertwined.

Language is a lot more than parsing syntax, whatever your thoughts are on the matter, even LLMs are clearly doing more than that. Are there any experiments where subjects had severe cognitive deficiencies and language in its full breadth or maybe i should say communication ? came out unscathed ?

The chris experiments don't seem to go into much detail in that front.

Re:Using LLMs to Understand Language Processing in the Human... by foobarqux on Wednesday March 26, 2025 @01:54PM (#43484856)
I just gave one: "Chris". Here's Chomsky describing the "Chris"-experiments ([1]) as part of a broader answer about how language is distinct from general cognition which I paraphrased above.

> That doesn't contradict the argument that “thinking” and “language processing” are not two sequential or clearly separated modes in the brain but deeply intertwined.

It's not an argument, it's an assertion, that is, in fact, contradicted by the experimental evidence I described (Moro and "Chris"). Of course they are "deeply intertwined" but because of the evidence it's probably an interface between two distinctive systems rather than one general system doing two tasks.

https://youtu.be/Rgd8BnZ2-iw?si=_UdBfG1InmpjbDPB&t=6735

Re:Using LLMs to Understand Language Processing in the Human... by og_kalu on Wednesday March 26, 2025 @02:08PM (#43485012)
Like i said, these experiments stop at a vague 'Chris can still learn languages'. No comment on actual proficiency or testing. For all i know i can't have a meaningful conversation with this guy beyond syntactically correct speech. Or maybe the best proficiency he's ever managed is still pretty poor compared to the average human. I have no idea.

There's no contradiction because i never argued/asserted the brain didn't have parts tuned for language, which is really all this experiment demonstrates.

Re:Using LLMs to Understand Language Processing in the Human... by foobarqux on Wednesday March 26, 2025 @02:29PM (#43485271)
It's irrelevant to the experiment: he could learn synthetic language with human-like grammar and could not learn synthetic languages with non-human-like grammar. Regular people could solve the non-human-like languages with difficulty. Because his language ability is much higher than his general problem solving ability it gives strong evidence that 1. human language capacity a special function, not a general purpose cognitive function and 2. it obeys a certain structure.

> There's no contradiction because i never argued/asserted the brain didn't have centers tuned for language, which is really all this experiment demonstrates.

I don't know what you are trying to say then.

Re:Using LLMs to Understand Language Processing in the Human... by og_kalu on Wednesday March 26, 2025 @02:51PM (#43485523)
>Because his language ability is much higher than his general problem solving ability

I don't see how you can say his language ability is much higher than his general problem solving ability if you don't know what proficiency of language he is capable of reaching.

When you are learning say English as a second language, there are proficiency tiers you get assigned when you get tested - A1, A2 etc

If he's learning all these languages but maxing out at A2 then his language ability is only slightly better than his general problem solving ability.

This is the point i'm trying to drive home. Maybe it's because i've been learning a second language for a couple years and so i see it more clearly but saying 'he learned x language' says absolutely nothing. People say that to mean anything from 'well he can ask for the toilet' to 'could be mistaken for a native'.

>I don't know what you are trying to say then.

The brain has over millions of years been tuned to speak languages with certain structures. Deviating from these structures is more taxing for the brain. True statement. But how on earth does that imply the brain isn't 'thinking' for the structures it is used to ? Do you say you did not think for question 1 just because question 2 was more difficult ?

Re:Using LLMs to Understand Language Processing in the Human... by foobarqux on Wednesday March 26, 2025 @03:28PM (#43486047)
As I said it's not relevant but if you wanted to know you could put in the bare minimum of effort into doing your own research. From Smith and Tsimpli's "The Mind of a Savant": "On the [Gapadol Reading Comprehension Test] Christopher scored at the maximum level, indicating a reading comprehension of 16 years and 10 months". They describe the results of a bunch of other language tests, where he scores average to above average, including his translations of passages from a dozen different languages.
Re:Using LLMs to Understand Language Processing in the Human... by hakaneskici on Tuesday March 25, 2025 @04:22PM (#43475508)
> current models are more than "stochastic parrots"

I believe the same, and also I'm willing to accept that the human brain can intentionally operate in an stochastic parrot mode.

Some people have the ability to fluently speak non-stop, completely impromptu. I wonder if it's similar to an LLM pipeline, where there'a constant stream of thoughts being generated based on very recent context, which are then passed through various output filters.

Re:Using LLMs to Understand Language Processing in the Human... by iamleppert on Tuesday March 25, 2025 @05:22PM (#43476128)
That certainly would explain how Trump can rattle on for hours and hours, despite a notable lack of high quality training data.
Re:Using LLMs to Understand Language Processing in the Human... by foobarqux on Tuesday March 25, 2025 @04:49PM (#43475801)
The correlations are 0.25-0.5, which is quite poor (Gaussian distribution plots with those correlations look like noise). That's before analyzing the methodology and assumptions.
Re:Using LLMs to Understand Language Processing in the Human... by bravura on Tuesday March 25, 2025 @05:24PM (#43476145)
Correlation of 0.25-0.5 being poor is very problem dependent.

For example, in difficult perceptual tasks ("can you taste which of these three biscuits is different" [one biscuit is made with slightly less sugar]), a correlation of 0.3 is commonplace and considered an appropriate amount of annotator agreement to make decisions.

Re:Using LLMs to Understand Language Processing in the Human... by foobarqux on Tuesday March 25, 2025 @05:48PM (#43476373)
Yes for certain things like statistical trading (assuming some kind of "nice" Gaussian-like distribution) where you have lots of trades and just need to be more right than wrong it's probably useful.

Not here though, where you are trying to prove a (near) equivalence.

Re:Using LLMs to Understand Language Processing in the Human... by ogogmad on Tuesday March 25, 2025 @04:50PM (#43475812)
Isn't correlation a wrong way of measuring statistical dependence anyway? I thought that entropy-based method like Relative Entropy did this better.
Re:Using LLMs to Understand Language Processing in the Human... by foobarqux on Tuesday March 25, 2025 @04:58PM (#43475896)
Yeah, Taleb has some pretty convincing arguments that correlation is generally bad/misleading and entropy is better.
Re:Using LLMs to Understand Language Processing in the Human... by devmor on Tuesday March 25, 2025 @05:38PM (#43476290)
I do not understand what you find convincing about this that changes your mind.

We have a closed system that we designed to operate in a way that is similar to our limited understanding of how a portion of the brain works, based on how we would model that part of the brain if it had to traverse an nth-dimensional array. We have loosely observed it working in a way that could roughly be defined as similar to our limited understanding of how a portion of the brain works given that limitation that we know is not true of the human brain, with a fairly low confidence level.

Even if you put an extreme level of faith into those very subpar conclusions and take them to be rigid... That does not make it actually similar to the human brain, or any kind of brain at all.

Re:Using LLMs to Understand Language Processing in the Human... by robmccoll on Tuesday March 25, 2025 @05:50PM (#43476392)
I view this as further evidence that we less different from "stochastic parrots" than we would like to believe.
Re:Using LLMs to Understand Language Processing in the Human... by corysama on Tuesday March 25, 2025 @07:20PM (#43477142)
<user corysama squawks indignantly and flies away...>
Re:Using LLMs to Understand Language Processing in the Human... by flir on Tuesday March 25, 2025 @05:57PM (#43476457)
Galaxies, hurricanes, whirlpools. Black Spleenwort, Barnsley Fern. Power laws every bloody where.

(My utterly uninformed knee-jerk reaction here, but even if I was a true believer I don't think I'd reach for "compelling".)

Re:Using LLMs to Understand Language Processing in the Human... by whymeogod on Tuesday March 25, 2025 @07:22PM (#43477163)
natural consequence when results are the sum of random processes.

if you add random rolls, you get a gaussian, thanks central limit theorem.

if you sum them, you get a lognormal distribution, which approximates a power law up to a cutoff

Re:Using LLMs to Understand Language Processing in the Human... by hatthew on Tuesday March 25, 2025 @08:26PM (#43477586)
Yeah, I have always firmly maintained that there is less fundamental difference between LLMs and human brains than most people seems to assume.

Going a bit further, I'll speculate that the actions made by a human brain are simply a function of the "input" from our ~5 senses combined with our memory (obviously there are complications such as spinal reflexes, but I don't think those affect my main point). Neural nets are universal function approximators, so can't a sufficiently large neural net approximate a full human brain? In that case, is there any merit to saying that a human "understands" something in a way that a neural net doesn't? There's obviously a huge gap between the two right now, but I don't see any fundamental difference besides "consciousness" which is not well defined to begin with.

Re:Using LLMs to Understand Language Processing in the Human... by dboreham on Tuesday March 25, 2025 @10:03PM (#43478101)
Ah, so there's two of us now :)
Re:Using LLMs to Understand Language Processing in the Human... by AlotOfReading on Tuesday March 25, 2025 @09:57PM (#43478065)
The UAT is a pretty weak result in practice. A lot of systems have the same property, and most of them are pretty poor approximators in practice. It may very well be that no reasonable amount of computing power allows approximating the "function of consciousness". Plus, if you're a certain kind of dualist the entire idea of a compact, smooth "consciousness" function may be something you reject philosophically.
Re:Using LLMs to Understand Language Processing in the Human... by hatthew on Tuesday March 25, 2025 @10:58PM (#43478411)
I agree there are issues with the UAT, but I feel like my conclusion is still valid: a neural net, given the memories and senses that a humans has, is capable of approximating a human's response accurately enough to be indistinguishable from another human, at least to another human.

I philosophically reject the notion that consciousness is an important factor here. The question of whether or not you have a consciousness doesn't affect what I take away from this conversation, and similarly the question of whether an AI has a consciousness doesn't affect what I take away from my actions with it. If the (non-)existence of others' consciousnesses doesn't materially affect my life—and we assume that it's a fundamentally unanswerable question—why should I care other than curiosity?

Re:Using LLMs to Understand Language Processing in the Human... by suddenlybananas on Wednesday March 26, 2025 @04:11AM (#43479935)
>a neural net, given the memories and senses that a humans has, is capable of approximating a human's response accurately enough to be indistinguishable from another human, at least to another human.

That doesn't remotely follow from the UAT and is also almost certainly false.

Re:Using LLMs to Understand Language Processing in the Human... by hatthew on Wednesday March 26, 2025 @06:59PM (#43488459)
I am not saying that follows from the formal UAT, I'm saying that in general, if we assume that the human brain is simply producing outputs in response to inputs, I have no reason to believe that a neural net couldn't be constructed to approximate the same outputs given the same inputs. And specifically, the threshold of "good enough approximation" I chose is effectively just a stronger form of the turing test. AI has arguably already passed a weaker form of the turing test (most people in the world cannot tell the difference between a human-written and an AI-written article, for example).
Re:Using LLMs to Understand Language Processing in the Human... by __loam on Wednesday March 26, 2025 @05:35AM (#43480354)
> I have always firmly maintained that there is less fundamental difference between LLMs and human brains than most people seems to assume.

What is your basis for this? Do you have any evidence or expertise in neuroscience to be able to make this claim?

> Neural nets are universal function approximators, so can't a sufficiently large neural net approximate a full human brain?

We do not understand the brain well enough to make this claim.

> but I don't see any fundamental difference besides "consciousness" which is not well defined to begin with.

Yeah besides the gaping hole in our current understanding of neuroscience, you have some good points I guess.

Re:Using LLMs to Understand Language Processing in the Human... by hatthew on Wednesday March 26, 2025 @06:50PM (#43488369)
Computers can compute anything, with the only limitations being time, space, energy, programming, etc. So far, it seems like all of those things can be increased without bound. I see no reason to believe that there will be some fundamental limitation that prevents us from improving computers to the point that they can compute the equivalent of a human brain. Yes I agree we don't understand the human brain well enough yet, but empirically we can observe that AI agents and human agents behave in similar ways. Sure, it may not end up being a complete copy of a human brain, but if the inputs and outputs are the same, does it make a difference?
Re:Using LLMs to Understand Language Processing in the Human... by visarga on Wednesday March 26, 2025 @01:22AM (#43479108)
> I view this as compelling evidence that current models are more than "stochastic parrots,"

More evidence against "stochastic parrots"

- zero shot translation, where LLMs can translate between unseen pairs of languages

- repeated sampling of responses from the same prompt - which shows diversity of expression with convergence of semantics

- reasoning models - solving problems

But my main critique is that they are better seen as pianos, not parrots. Pianos don't make music, but we do. And we play the LLMs on the keyboard like regular pianos.