>>>> They cannot say "I dont know"

>>> If they were trained with more uncertain content, perhaps they'd be better at expressing uncertainty as well.

>> (me) If you ask ChatGPT a question, and tell it to either respond with the answer or "I don't know", it will respond "I don't know" if you ask it whether you have a brother or not.

> This has nothing to do with thinking and everything to do with the fact that given that input the answer was the most probable output given the training data.

First of all, my claim was in response to "They cannot say 'I dont know'" and "perhaps they'd be better at expressing uncertainty".

ChatGPT can say "I don't know" if you ask it to.

Regarding whether LLMs are lookup tables, I responded to that in more detail elsewhere under this post:

https://news.ycombinator.com/item?id=39501611

And your post was the most probable output of your mind process given your experiences. The only self-evident difference is the richness of your experience as compared to LLMs.

No the self evidence difference is that the brain is equipped with many more models than simply language. Language is one of the ways we express the composite output of many models, emotion being a key other which has no need for language to exist.

This is why it is a fallacy to think an LLM contains anything other than the textual descriptions of our higher level thinking, and why LLM alone will only ever parrot intelligence.

> why LLM alone will only ever parrot intelligence.

Can you design a text only test that will differentiate between real intelligence and the parroted kind?

The "language" you're talking about is not the same as the "language" of large language models. Regardless of how many models the brain has, they must all be comparable using some common metric in order for attention, processing and processing to work. This type of common substrate is what LLMs operate on, and is how multimodal LLMs work.