GPT-4 performs better at Theory of Mind tests than actual humans

Re:GPT-4 performs better at Theory of Mind tests than actual... by amluto on Sunday April 30, 2023 @07:51PM (#35767633)

I’m not convinced that these are particularly useful tests of very large language models like GPT-4. These models have been trained on enormous amounts of text, including theory of mind tests. And the example scenario in the paper is very simple and likely related to things that the models in question have seen before by a simple substitution.

So I would actually be a bit surprised if the models failed this type of test regardless of whether they have any understanding of theory of mind.

Note that I’m making no claim about how intelligent or understanding of humans or neurotypical these models are. My claim is that it’s very hard to tell given how incredibly knowledgeable the models are.

[ Parent | on HN ]

Re:GPT-4 performs better at Theory of Mind tests than actual... by nopassrecover on Sunday April 30, 2023 @09:01PM (#35768032)

It's interesting how quickly the Chinese Room has moved from thought experiment to reality https://en.wikipedia.org/wiki/Chinese_room

[ Parent | on HN ]

Re:GPT-4 performs better at Theory of Mind tests than actual... by amluto on Sunday April 30, 2023 @10:58PM (#35768643)

From the description in the Wikipedia article, we’ve already reached the point where it’s quite interesting. An nVidia card and an iPhone are pretty clearly not intelligent (although the iPhone tries a bit harder).

But these same devices, running the right software and given the right parameters, do something that seems impressively close to intelligence. And I suspect that my brain, with its software somehow removed, would not seem very intelligent.

On the other hand, a human baby will spontaneously generate language and intelligence, and I haven’t heard of a language model doing that. (The experiment has, sadly, been done. https://en.m.wikipedia.org/wiki/Nicaraguan_Sign_Language )

[ Parent | on HN ]

Re:GPT-4 performs better at Theory of Mind tests than actual... by satvikpendem on Sunday April 30, 2023 @11:02PM (#35768663)

As someone that studied philosophy as well as computer science in college and had to hear many times people asking why I study philosophy at all, this is why.

Philosophy often asks questions that are not yet reached by current technology but will be soon enough.

[ Parent | on HN ]