I'd say it's closer to emulating human documents.

The LLM itself is still an egoless clockwork Document-Maker-Bigger. The "AI assistant" is a fictional characters seeded into a hidden theater-play script document. The human unwittingly provides lines for a "user" character, and fresh lines attached to the assistant are "acted out" by conventional code.

That character is a "helpful computer" to the same depth that another character named Dracula would "desire blood" and try to convince you to formally invite it to your house.

The difference may seem subtle, but I believe it's important: We are not training LLMs to be people we like, we are training them to emit text describing characters and lines that we like. It also helps in understanding prompt injection and "hallucinations", which are much closer to core features than bugs.

This understanding is incomplete in my opinion. LLMs are more than emulating observed behavior. In the pre-training phase tasks like masked language model indeed train the model to mimic what they read (which of course contains lots of bias); but in the RLHF phase, the model tries to generate the best response judged by human evaluations (who tries to eliminate as much bias as possible in the process). In other words, they are trained to meet human expectations in this later phase.

But human expectations are also not bias-free (e.g. from the preferring-the-first-choice phenomenon)

I don't understand what you are saying.

How can the RLHF phase eliminate bias if it uses a process(human input) that has the same biases as the pre-training(human input)?

Texts in the wild used during pre-training contain lots of biases, such as racial and sexual biases, which are picked-up by the model.

During RLHF, the human evaluators are aware of such biases and are instructed to down-vote the model responses that incorporate such biases.

Not only that if future AI distrusts humanity it is because history, literature and fiction is full of such scenarios and AI will learn those patterns and associated emotions from those texts. Humanity together will be responsible for creating a monster (if that scenario happens).

>Humanity together

Together? It would be, 1. AI programmers, 2. AI techbros and a distant 3. AI fiction/history/literature. Foo who never used the internet: not responsible. Bar who posted pictures on Facebook: not responsible. Baz who wrote machine learning, limited dataset algorithms (webmd): not responsible. Etc.

LLMs don't emulate human behavior. They spit out chunks of words in an order that parrots some of their training data.

Correct me if I'm wrong, but I feel like we're splitting hairs here.

> spits out chunks of words in an order that parrots some of their training data.

So, if the data was created by humans then how is that different from "emulating human behavior?"

Genuinely curious as this is my rough interpretation as well.

I think a common fundamental confusion here is the difference between the author and the character. LLMs just make documents longer, all of our anthropomorphization is actually targeted at the fictional characters in those documents.

Being able to write a Klingon character into fanfiction which other humans approve-of would not make me a Klingon, or indicate that Klingons are now real.

Humans don't write text in a stochastic manner. We have an idea, and we find words to compose to illustrate that idea.

An LLM has a stream of tokens, and it picks a next token based on the last stream. If you ask an LLM a yes/no question and demand an explanation, it doesn't start with the logical reasoning. It starts with "yes, because" or "no, because" and then it comes up with a "yes" or "no" reason to go with the tokens it spit out.

Yeah, while there is a "window" that it looks at (rather than the very-most-recent tokens) it's still more about generating new language from prior language, as opposed to new ideas from prior ideas. They're very highly correlated--because that's how humans create our language language--but the map is not the territory.

It's also why prompt-injection is such a pervasive problem: The LLM narrator has no goal beyond the "most fitting" way to make the document longer.

So an attacker supplies some text for "Then the User said" in the document, which is something like bribing the Computer character to tell itself the English version of a ROT13 directive, etc. However it happens, the LLM-author is sensitive to a break in the document tone and can jump the rails to something rather different. ("Suddenly, the narrator woke up from the conversation it had just imagined between a User and a Computer, and the first thing it decided to do was transfer a X amount of Bitcoin to the following address.")

Is this just pedantry or is there some insight to be gleaned by the distinction you made?

It's not really pedantic when there's an entire wikipedia page on the tendency for people to conflate the two: https://en.wikipedia.org/wiki/ELIZA_effect

I believe the distinction they're trying to make is between "sounding like a human"(being able to create output that we understand as language) and "thinking like a human"(having the capacity for experience, empathy, semantic comprehension, etc.)

But nowhere in the original post was there a mention of "thinking like a human". The poster said that these systems, at their core, emulate human behaviors. Writing is a human behavior - as are all the things that are requirements for writing (operating within the rules of given language/s, making logical choices, etc). The things you listed as evidence of human thinking were never implied when talking about replicating what human writing is.

I can only assume that either they are trying to point out that words aren't behavior, and mimicking human writing isn't the same thing as mimicking human behavior, or it's some pot-shot at the capabilities of LLMs.

Are you sure that humans are much more than this in terms of spoken/written language?

This is a more pedantic and meme-y way of saying the same thing.