Of course LLMs don't store facts. They only "experience" the world through text tokens, so at best they can store and process information about text tokens, and any information that can be inferred from those.

But that's exactly what philosophy has been arguing about regarding humans since at least Descartes's Evil Demon (the 17th century version of the brain in a vat). Humans don't know anything about "reality", they only know what their senses are telling them. Which is at best a very skewed and limited view of reality, and at worst completely wrong or an illusion.

We perceive the world through more facets than an LLM, but fundamentally we share many of their limitations. So if someone says "LLMs don’t store facts", I find "neither do humans" a very reasonable answer, even if its only purpose is to show that "can it store facts" is a bad metric.

Of course the more productive part to argue about is the "are facts and language patterns really mutually exclusive", which leads right into "if you had to design an efficient token predictor, would it do 'dumb' math like a markov chain, or would your design include some kind of store of world knowledge? Can you encode a knowledge store in a neural network? And if you can, how can you tell that LLMs don't do that internally?"