The whole input of your current session is fed into the model, that's why it tricks you that it "KNOWS" like human would know, in reality this is a lot of data, computation and statistics without any reasoning, that's why there is a lot of examples showing it contradicts itself in the same paragraph, because it doesn't know the meaning of the words its using in the same way as humans do, it only knows probabilities of words in sequences.

LLMs are trained exclusively on text, which means they lack crucial context behind the meaning of sentences. The universe of information outside of pure text - vision, sound, etc is completely unknown to it.

LLMs are basically the aliens in blindsight. They have a superhuman ability to memorize the context of words it has seen and generalize to new contexts, but it can never be perfect because it's working on incomplete information.

> it can never be perfect because it's working on incomplete information.

Unlike you?

Yeah, that's the actual bit that baffles me about ChatGPT still. Producing coherent, fluent text is alright, but we could already sort of do that 20 years ago with markov models or even just grammars (see Chomsky).

Understanding text in the depth that ChatGPT (and GPT-3) appear to understand the prompts is something entirely different and has to my knowledge never been archieved before the current architectures.

There is a lot of knowledge encoded into the model, but there's a difference between knowing what a sunset is because you read about it on the internet vs having seen one.