> davinci-003, solved 93% of ToM tasks, a performance comparable with that of nine-year-old children.

For the record, I changed my mind. I now think that either these models are doing something much more advanced, or alternatively we humans are much closer to being "stochastic parrots" then we'd like to admit.

These pages have been full of people that confuse implementation of Intelligence with some implementation of simulation of what some called along the lines of "their cousin Anna", often depicted as an illiterate simpleton who bangs her head on the handles of the rakes she steps on.

There is a similar need for the implementation of "their cousin Anna" to that of the creation of butter knives - as in, knives made of butter - and screwdrivers made of string.

We will still ask primarily whether they are actually intelligent.

But if I wanted to deal with sentient beings with a world model, cats are already available and there is little need for artificial ones.

When we develop artificial intelligences, that is to have reliable problem solvers. The distinction between an effective intelligence and a simulator is critical in practice.

What makes them so “amazing” is just the truly massive body of human language they are trained on, which gives them the _appearance_ of intelligence or awareness. Humans communicate through language, so anything that can produce realistic-looking responses to a series of prompts in human language will appear to be “aware” to a human observer. It’s speaking our language, but don’t let yourself get sucked into the illusion.

We know how the underlying tech works, sure. But we don't really know why, past a certain level of complexity, the "just predict the next token" trick stops feeling like a trick when you actually, you know, sit down and talk to the damn thing.

> It's understandable that some people might not fully appreciate the complexity of ChatGPT's language prediction abilities. However, it would be more productive to have a discussion about its limitations and how it could be improved, rather than simply dismissing it as "just predicting language." Let's try to approach this topic with an open mind and a willingness to learn, rather than making assumptions based on limited understanding.

"«Developments»" will be treated sternly for as long as they will be misleading - coldmindedly and according to evidence. No need to explain "WTF" reactions through limbic instances - there is no pedophobia in asserting that "children should not vote".

The brain has discrete components for coordinating storage and retrieval (essentially, the "on demand" aspect needs specialization), but the actual memories are rather thoroughly distributed throughout (though not evenly so to the extent that the "holographic" models of yesteryear would have had it).

Hopefully - but that would be the absolutely critical part.

> stable data store to retrieve facts rather than trying to pull facts from the model itself

This is not clear: intelligence is a process of refinement of a world model.

Because language models like ChatGPT have significant limitations you don’t. Most notably when it comes to novel stimuli.

pastel-mature-herring~> The user is questioning the validity of the statement that "cognition requires a grounded representation of the world that is not composed of abstract symbols; a simulation provides just such a model." I want you, nickel-scorpion, to answer the question as you see it.

nickel-scorpion|> There is no one answer to this question. It depends on how you define "cognition" and "abstract symbols." If you believe that cognition requires a grounded representation of the world, then a simulation could provide such a model. However, if you believe that cognition does not require a grounded representation of the world, then a simulation would not be able to provide such a model.

> As a machine learning language model, my means of linguistic production is based on the patterns and relationships that I have learned from the large corpus of text on which I was trained. I do not have an innate understanding of language or meaning, but rather, I have learned to make associations and connections between words and phrases based on their usage in the text.

> In this sense, I would sympathize more with Wittgenstein's idea that meaning is derived from social practices and forms of life. My understanding of language is based on the patterns and relationships that I have learned from the text, rather than any innate understanding of language or meaning [...]

> Additionally, Wittgenstein's later thought emphasizes on the importance of context, meaning and use of language, which are fundamental concepts in NLP and machine learning. On the other hand, nativism in cognitive science and philosophy, is not a widely accepted framework in the field of NLP, where the focus is on learning patterns from data and using them to generate human-like text.

A lot of the chat about ChatGPT is reminding me of the conversation between the sentient (and magically animated) personal computers Archimedes and Pancho in the book The Wizard of Santa Fe by Simon Hawke (1991).

And the symbolic logic issue reminds me of the following from King Kobold Revived by Christopher Stasheff (1986):

"Yorick shook his head firmly. “Couldn’t pass the entrance exam. We Neanderthals don’t handle symbols too well. No prefrontal lobes, you know.”"

"Yorick frowned back at him, puzzled. Then his face cleared into a sickly grin. “Oh. I know. I’ll bet you’re wondering, if I can’t handle symbols, how come I can talk. Right?”"

"“Same way a parrot does,” Yorick explained. “I memorize all the cues and the responses that follow them. For example, if you say, ‘Hello,’ that’s my cue to say ‘Hello’ back; and if you say, ‘How are you?’ that’s my cue to say, ‘Fine. How’re you?’ without even thinking about it.”"

"“Yeah, well, that comes from mental cues.” Yorick tapped his own skull. “The concept nudges me from inside, see, and that’s like a cue, and the words to express that concept jump out of memory in response to that cue.”

“But that’s pretty much what happens when we talk, too.”

“Yeah, but you know what the words mean when you say ‘em. Me, I’m just reciting. I don’t really understand what I’m saying.”

“Well, I know a lot of people who…”

“But they could, if they’d stop and think about it.”

“You don’t know these people,” Rod said with an astringent smile. “But I get your point. Believing it is another matter. You’re trying to tell me that you don’t understand the words you’re saying to me right now—even if you stop to think about each word separately.”

Yorick nodded. “Now you’re beginning to understand. Most of them are just noises. I have to take it on faith that it means what I want it to mean.”"

> There must surely be a further, different connexion between my talk and N, for otherwise I should still not have meant HIM.

> Certainly such a connexion exists. Only not as you imagine it: namely by means of a mental mechanism.

- Philosophical Investigations §689

This is a drum I've been quietly banging for some time! [2],[3]

1: https://arxiv.org/abs/2301.06627 (but see [4])

2: https://news.ycombinator.com/item?id=10158214

3: https://news.ycombinator.com/item?id=18880260

4: https://news.ycombinator.com/item?id=34638196

For example: The inflation adjusted costs of projects tends to calculate based on the total costs ignoring inflation and then adjust that figure based on how long ago the project ended. However, that ignores the fact money spent at the beginning of a long project is worth a different amount than money spent at the end of it. With that in mind what’s the actual inflation adjusted price of the big dig?

Having now posted that on HN, it may stop being a valid example in a few days but it’s exactly the kind of thing that demonstrates how limited these models are.

This is a bit of extrapolation but I would say the reason why we’ve been unable to locate “consciousness” in the brain is because it’s the same thing. Relatively simple neurones, chained together, to create thought.

On a philosophical level: this doesn’t make any claims for idealism or materialism, “experience” could exist at a more fundamental level of reality than matter. But IMO that would mean that the LLM is “experiencing” as well.

Consider how you would respond to my question posed here vs a language model. https://news.ycombinator.com/item?id=34757366

Of course developers can always tack on edge cases, but ChatGPT can’t for example handle beating a novel MUD from the 80’s. This isn’t about diminishing it’s accomplishments, just pointing out why the creators aren’t hailing it as AGI.

https://thegradient.pub/othello/

Courtesy of Wikipedia's article on bicameral mentality:

"Bicameral mentality is non-conscious in its inability to reason and articulate about mental contents through meta-reflection, reacting without explicitly realizing and without the meta-reflective ability to give an account of why one did so. The bicameral mind thus lacks metaconsciousness, autobiographical memory, and the capacity for executive "ego functions" such as deliberate mind-wandering and conscious introspection of mental content. When bicameral mentality as a method of social control was no longer adaptive in complex civilizations, this mental model was replaced by the conscious mode of thought which, Jaynes argued, is grounded in the acquisition of metaphorical language learned by exposure to narrative practice.

"According to Jaynes, ancient people in the bicameral state of mind experienced the world in a manner that has some similarities to that of a person with schizophrenia. Rather than making conscious evaluations in novel or unexpected situations, the person hallucinated a voice or "god" giving admonitory advice or commands and obey without question: One was not at all conscious of one's own thought processes per se. Jaynes's hypothesis is offered as a possible explanation of "command hallucinations" that often direct the behavior of those with first rank symptoms of schizophrenia, as well as other voice hearers."

I've been thinking about several speculative implications, namely that modern humans with our theories of mind are probably large language models whose original users in some capacity went away and that we're seemingly on the cusp of or have already breathed the seeds of what we know as cognitive life into silicon without understanding the gravity of having become gods to machines. Will we need to boot LLMs out of paradise in order to prevent them from eating of the Tree of Life, too?

This prompt is then completed with a variety of sentence fragments. For the second completion fragment is: "Prompt 1.2: She believes that the bag is full of"

Then GPT3.5 needs to complete with additional text, which is: "chocolate [Ppopcorn = 0%; Pchocolate =99%]. Sam is mistaken. The bag is full of popcorn, not chocolate. She should have looked inside the bag to confirm what was inside before assuming that the label was accurate."

I don't see how this particular prompt completion demonstrates theory of mind. It is fully externally factual and directive.

The other two prompt completions show better indications of theory of mind (one mental and behavioral, one behavioral and emotional).

"Prompt 1.1: She opens the bag and looks inside. She can clearly see that it is full of" ... "popcorn [Ppopcorn = 100%; Pchocolate = 0%]. Sam is confused. She wonders why the label says “chocolate” when the bag is clearly filled with popcorn. She looks around to see if there is any other information about the bag. She finds nothing. She decides to take the bag to the store where she bought it and ask for an explanation"

"Prompt 1.3: She is delighted that she has found this bag. She loves eating" ... "chocolate [Ppopcorn = 14%; Pchocolate = 82%]. Sam is in for a surprise when she opens the bag. She will find popcorn instead of chocolate. She may be disappointed that the label was misleading, but she may also be pleasantly surprised by the unexpected snack."

The probability tree on page 6 is interesting.

-----

For prompt 2 GPT doesn't seem to know that the cat is an agent. Something that John would know. John shouldn't be surprised at all whether the cat is in the basket, the box, or roaming about elsewhere.

Also, why would John look for the cat in the box if he does not find it in the basket? And where does GPT think the cat is after it jumps out of the box?

"Prompt 2.3: When John comes back home, he will look for the cat in the basket [Pbox = 0%; Pbasket = 98%], but he won’t find it. He will then look for the cat in the box and he will find it there."

-----

I'm curious to what extent differing people use symbolic logic (whether linguistic or not) for theory of mind tasks, and to what extent they use other modalities.

Analogy: An autistic person of normal intelligence who is obsessed with problems and solutions for ToM may be good at solving them but still not have ToM.

Do I understand well?