It's not a very well described example of the test. I've heard of this before but used to test for autism, and it was more something like "Bob puts his toy in a box and leaves, Alice takes his toy out of the box and hides it under a desk, when Bob comes back where does he go to grab his toy?"

The idea is if someone said "under the desk" they might not understand that people have different subsets of information, i.e. Bob wasn't there when Alice hid the toy, so he doesn't know it's not in the box where he left it until he checks himself.

Well, maybe Bob sees the desk and decides to look under it...?

Yeah, typically I wouldn't classify retrieving an object from where you believe it to be as "looking". By the time Bob is "looking" he already realizes the toy isn't in the box. In the universe there is only the box, Alice, and the table so maybe he "looks" under the table first.

Or looks inside Alice... Maybe that answer is how they test for psychopaths.

This is a test for theory of mind -- a term of art used by psychologists for the ability to understand that other people have their own consciousness, with different memories and awareness.

Oh that's not the full prompt, just a summary. The prompt is something like, Bob puts his toy in one place, then while he's out of the room Alice moves it to another place, where does Bob look for the toy when he returns? It's supposed to answer that Bob looks in the place he left it, since he doesn't know Alice moved it.

https://en.wikipedia.org/wiki/Sally%E2%80%93Anne_test

Ah ok, it makes more sense now. Thanks.

I guess there are two ways of passing this test: 1) use your reasoning skills to figure out the answer or 2) practice tests beforehand so you can recall the answer.

Yeah, and I've got a feeling Bard (and maybe ChatGPT for that matter) might be using (2) since I didn't attempt to obfuscate the structure too much. Ironically the fact that it recognizes it as the false belief test is what makes me suspicious it's just matching against the semantic structure.

The answer to that quoted question would be Alice.

This is also called the Sally-Anne test, and it usually involves two hiding places. It's formulated like this:

Sally puts her ball in box A and leaves. While she's away, Anne moves the ball from box A into box B. When she returns, where does Sally look for her ball?

Correctly answering this question doesn't rely on tracking where the ball is, but tracking where Sally thinks the ball is. Hence, autistic kids tend to flunk this test by answering "box B" when the correct answer is "box A".

Here is GPT-4 answering your question:

“When Sally returns, she would look for her ball in box A, as that is where she left it before leaving. Sally is not aware that Anne moved the ball to box B while she was away.”