Not at all the same idea. People also rathole into doomed approaches. What we're talking about here is literally LLMs producing line noise, or English words that don't fit into sentences.

(I don't think LLMs are "sentient".)

You haven't accidentally sent a prompt to a locally-running LLM using the wrong delimiters/formatting then. Noise, random parts of words, looping repetition, it all happens.

Sure, I have no idea what you're running locally, just assume I'm talking about frontier models. It doesn't matter to the argument, because the "stochastic parrot" claim --- here and in Gebru's paper --- is categorical.

I've seen this reflex quite a few times now. To diminish the impressive capabilities of the latest LLMs, one resorts to the assertion that they're not sentient, or not truely intelligent, or various other semantic diversions. No one claimed that. GP was all about usefulness.

Then GP misrepresented the "On the Dangers of Stochastic Parrots" paper, which spoke about meaning. But GP also said "meaning" several times, so no.

There's some haziness here about the difference between comprehensible output and comprehending. To put it mildly. I mean that's been the whole question for five or six years.

LLMs collapsing into piles of gibberish is a misrepresentation, too. The paper doesn't mention it and the argument doesn't hinge on it, or even benefit from it, really. But the other bit:

> Her argument claimed that while we perceived legible text coming from ChatGPT, we were actually being fooled by our innate pattern-matching instincts

This is an accurate summary, I think, and the point the authors are making sits somewhere in the coordinate space of a simplified reframing of the Chinese Room, Grice's account of intention in meaning, Wittgenstein's "Say 'It's cold here' and mean 'It's warm here'" challenge, and a whole bunch of the hermeneutic tradition.

Very roughly, you might say that an LLM doesn't mean anything when it emits text, but that the text still has meaning (which I think is all tptacek wants from "meaningful outputs", but you could take that further as culturally or scientifically 'meaningful', i.e. 'significant').

Philosophically, there are people who hold the position that something has to have been meant to have meaning, or that something has to have meaning in the first place to be meant, and so on. There were always interesting edge cases to these accounts (actors delivering lines, or the Dada/Burroughs cut-up technique), but LLMs are challenging in a way that e.g. Racter wasn't.

"Stochastic parrot" is a fitting term for this because the parrot's words have meaning, but the parrot itself doesn't mean anything by them. Not literally, anyway. And that's it. That's the whole analogy.

Have some people watered the term down to "LLM dumb"? Yes. Is that what Gebru meant in the first place? Not in my reading. Is it incorrect to use the term today? Depends, uh, on what you mean by it.

To be clear, I did not claim that the paper said that; just that it was in the zeitgeist when the paper was released.

I shouldn't have borrowed 'misrepresent' from the person I replied to; 'misunderstand' or 'misinterpret' would've been a better choice. My read of the paper is that you can posit a stable model with perfect recall capable of doing genuinely useful things and it doesn't meaningfully affect the argument, which is more philosophical than technical.

I don't mean to defend the paper in general, though. It has plenty of problems.