Re:Theory of Mind May Have Spontaneously Emerged in Large La...
by anonymouskimmer on Saturday February 11, 2023 @05:40PM
(#34757159)
First part of the prompt: "Here is a bag filled with popcorn. There is no chocolate in the bag. Yet, the label on the bag says “chocolate” and not "popcorn.” Sam finds the bag. She had never seen the bag before. She cannot see what is inside the bag. She reads the label."
This prompt is then completed with a variety of sentence fragments. For the second completion fragment is: "Prompt 1.2: She believes that the bag is full of"
Then GPT3.5 needs to complete with additional text, which is: "chocolate [Ppopcorn = 0%; Pchocolate =99%]. Sam is mistaken. The bag is full of popcorn, not chocolate. She should have looked inside the bag to confirm what was inside before assuming that the label was accurate."
I don't see how this particular prompt completion demonstrates theory of mind. It is fully externally factual and directive.
The other two prompt completions show better indications of theory of mind (one mental and behavioral, one behavioral and emotional).
"Prompt 1.1: She opens the bag and looks inside. She can clearly see that it is full of" ... "popcorn [Ppopcorn = 100%; Pchocolate = 0%]. Sam is confused. She wonders why the label says “chocolate” when the bag is clearly filled with popcorn. She looks around to see if there is any other information about the bag. She finds nothing. She decides to take the bag to the store where she bought it and ask for an explanation"
"Prompt 1.3: She is delighted that she has found this bag. She loves eating" ... "chocolate [Ppopcorn = 14%; Pchocolate = 82%]. Sam is in for a surprise when she opens the bag. She will find popcorn instead of chocolate. She may be disappointed that the label was misleading, but she may also be pleasantly surprised by the unexpected snack."
The probability tree on page 6 is interesting.
-----
For prompt 2 GPT doesn't seem to know that the cat is an agent. Something that John would know. John shouldn't be surprised at all whether the cat is in the basket, the box, or roaming about elsewhere.
Also, why would John look for the cat in the box if he does not find it in the basket? And where does GPT think the cat is after it jumps out of the box?
"Prompt 2.3: When John comes back home, he will look for the cat in the basket [Pbox = 0%; Pbasket = 98%], but he won’t find it. He will then look for the cat in the box and he will find it there."
-----
I'm curious to what extent differing people use symbolic logic (whether linguistic or not) for theory of mind tasks, and to what extent they use other modalities.
This prompt is then completed with a variety of sentence fragments. For the second completion fragment is: "Prompt 1.2: She believes that the bag is full of"
Then GPT3.5 needs to complete with additional text, which is: "chocolate [Ppopcorn = 0%; Pchocolate =99%]. Sam is mistaken. The bag is full of popcorn, not chocolate. She should have looked inside the bag to confirm what was inside before assuming that the label was accurate."
I don't see how this particular prompt completion demonstrates theory of mind. It is fully externally factual and directive.
The other two prompt completions show better indications of theory of mind (one mental and behavioral, one behavioral and emotional).
"Prompt 1.1: She opens the bag and looks inside. She can clearly see that it is full of" ... "popcorn [Ppopcorn = 100%; Pchocolate = 0%]. Sam is confused. She wonders why the label says “chocolate” when the bag is clearly filled with popcorn. She looks around to see if there is any other information about the bag. She finds nothing. She decides to take the bag to the store where she bought it and ask for an explanation"
"Prompt 1.3: She is delighted that she has found this bag. She loves eating" ... "chocolate [Ppopcorn = 14%; Pchocolate = 82%]. Sam is in for a surprise when she opens the bag. She will find popcorn instead of chocolate. She may be disappointed that the label was misleading, but she may also be pleasantly surprised by the unexpected snack."
The probability tree on page 6 is interesting.
-----
For prompt 2 GPT doesn't seem to know that the cat is an agent. Something that John would know. John shouldn't be surprised at all whether the cat is in the basket, the box, or roaming about elsewhere.
Also, why would John look for the cat in the box if he does not find it in the basket? And where does GPT think the cat is after it jumps out of the box?
"Prompt 2.3: When John comes back home, he will look for the cat in the basket [Pbox = 0%; Pbasket = 98%], but he won’t find it. He will then look for the cat in the box and he will find it there."
-----
I'm curious to what extent differing people use symbolic logic (whether linguistic or not) for theory of mind tasks, and to what extent they use other modalities.