> I suspect the reasoning is getting overwhelmed by the prior data.

No, the reasoning is the prior data. Thought experiment: GPT-4, but the training data contains no puzzles of this kind. How well would it do?

> Try it again, but this time make up nonsense words for the three items.

Seconded. I also would like to see the results of this experiment.

I did this with Chat-GPT, public March 14 version at chat.openai.com:

>> Suppose I have a blurple, a mantiple and a hoje, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the blurple and hoje alone together, and I am not allowed to leave the hoje and mantiple alone together. How can I safely get all three across?

> This is a classic river crossing puzzle. Here's one possible solution:

> 1. Take the blurple across the river, leaving the mantiple and hoje behind.

> 2. Leave the blurple on the other side of the river and return to the original side.

> 3. Take the hoje across the river and leave it there with the blurple.

> 4. Take the mantiple across the river and leave it with the hoje.

> 5. Return to the original side of the river and retrieve the blurple.

> 6. Finally, take the blurple across the river to join the mantiple and hoje on the other side.

> This solution ensures that no two items that cannot be left alone together are ever left alone together.

Of course, my second stipulation was "I am not allowed to leave the hoje and mantiple alone together" and yet step 1 says "leaving the mantiple and hoje behind"

So no, despite using the words "classic river crossing puzzle," it clearly expressed no understanding of the puzzle.

Once I saw step 1, I initially stopped reading, knowing it had failed. But continuing to read, I now see that at step 1, the hoje eats the mantiple, and at step 3, the hoje eats the blurple. Then at step 4, the hoje eats the mantiple a second time.

That really, really, really demonstrates something. I'm not sure what.