> And they only encoded it. They didn't learn it, and they don't know it.

People say stuff like this a lot, and I never know what to do with it. I'm not sure what specifically I'm supposed to get out of these distinctions.

I'm not sure either. An alien could make a similar argument about our brains made of meat and electricity.

Well, without bothering to get all epistemological about it, it's clear that LLMs doesn't know, say, the law in a way that a lawyer knows it, or physics in the way that a physicist knows it.

And that's enough to be a problem, given how people are trying to use LLMs. They're trying to use it like a human who "knows" (whatever that means). And whatever an LLM actually does, it doesn't do that.

My uncle doesn't know the law the way a lawyer knows it either, but that doesn't stop him from trying to use it like someone who does ;)

If the claim is only "LLMs don't work like humans" and "LLMs make a lot of factual and logical errors", then that seems to be correct as far as it goes, but it doesn't go very far.

> But most of the training corpus was not linguistically-sound gibberish; it was actual text. There was knowledge encoded in the words. LLMs are large language models - large enough to have encoded some of the knowledge with the words.

>Some of. And they only encoded it. They didn't learn it, and they don't know it. It's just encoded in the words. It comes out sometimes in response to a prompt. Not always, not often enough to be relied on, but often enough to give users hope.

And some of the pieces of "knowledge" in the training corpora were wrong, lies, or bullshit themselves.

Garbage in, garbage out.