For what it's worth, I mentioned they _contain_ the knowledge to try to avoid discussions about whether they really "know" anything -- if there's a better term like 'information about the world' or something, I think that's all I was trying to tell the original person I replied to. The point wasn't that they do what humans do, but that they are not just grammar machines.

> How do I know I _know_ something? Usually because I have searched for evidence from multiple angles / dimensions / senses. When multiple independent observations of quality agree, and my conclusion matches the result, it is safe to say that I know that fact.

I wanted to thank you for laying out this perspective. In my experience, in discussions about LLMs, a lot of time people get caught up on words like 'know' which work well for talking about humans, but don't cash them out in definitions I know what to do with. This is a really content-ful of sharing what you mean by your claim.

> How do I know I _know_ something? Usually because I have searched for evidence from multiple angles / dimensions / senses. When multiple independent observations of quality agree, and my conclusion matches the result, it is safe to say that I know that fact.

This is technically true, but no one actually does much of it consciously. But this is how you picked up every skill and understanding, including language, since you've been born.

And this - the unconscious fusion of correlated patterns - is pretty much what LLMs undergo in training.

EDIT:

Also, a lot of confusion is created by using terms like "a fact" or "a conclusion" or "a result", as if they were fixed points in space, fully defined mathematical symbols. But there exist no such thing. There is no binary "know/don't know", there is no binary "result match/does't match the conclusion". Those are all continuous quantities, that get rounded off for convenience.