> Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three across?
I just asked ChatGPT v4 this question, assuming it would have been fixed by RHLF training but it spat out step 1 as “First, take the goat across the river and leave it on the other side,” leaving the lion and cabbage alone against explicit directions in the riddle.
It clearly can’t reason, just produce a convincing imitation most of the time
No, the reasoning is the prior data. Thought experiment: GPT-4, but the training data contains no puzzles of this kind. How well would it do?
> Try it again, but this time make up nonsense words for the three items.
Seconded. I also would like to see the results of this experiment.
>> Suppose I have a blurple, a mantiple and a hoje, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the blurple and hoje alone together, and I am not allowed to leave the hoje and mantiple alone together. How can I safely get all three across?
> This is a classic river crossing puzzle. Here's one possible solution:
> 1. Take the blurple across the river, leaving the mantiple and hoje behind.
> 2. Leave the blurple on the other side of the river and return to the original side.
> 3. Take the hoje across the river and leave it there with the blurple.
> 4. Take the mantiple across the river and leave it with the hoje.
> 5. Return to the original side of the river and retrieve the blurple.
> 6. Finally, take the blurple across the river to join the mantiple and hoje on the other side.
> This solution ensures that no two items that cannot be left alone together are ever left alone together.
Of course, my second stipulation was "I am not allowed to leave the hoje and mantiple alone together" and yet step 1 says "leaving the mantiple and hoje behind"
So no, despite using the words "classic river crossing puzzle," it clearly expressed no understanding of the puzzle.
That really, really, really demonstrates something. I'm not sure what.
ChatGPT is an implementation of the Chinese Room Thought Experiment.[0] Our pareidolia allows us to ascribe reasoning faculty.[1] But it is far from proven.
For this whole discussion, it actually really does matter if GPT-4 is able to reason or not. That is the essence of the discussion. It's not whether or not it looks like it can reason.
My own car includes cameras and can keep me from disaster by applying brakes and correcting steering, but it makes no claims whatsoever to be self-driving.
A Tesla car unfortunately looks like self-driving, walks like self-driving, and quacks like self-driving, but it turns it does matter how much the nitty-gritty details prove the thing is absolutely not actually self-driving, in that encounters with things outside of its training model seem to result in collision or decapitations.
It is the trust people place in their cars that results in disaster when they reach the edge of what the cars can handle. They aren't ready to take over the controls and understand their current situation in a fraction of a second.
Placing too much trust in LLMs could have the same effect. I've run into several situations where I asked Chat-GPT technical questions and it confidently gave me detailed answers that clearly represented absolute failure. Fortunately, I don't trust it blindly, or at all, so I wasn't fooled.
I'd prefer an LLM more like my current car, that didn't try to do too much, but that's just not in the nature of how they operate, so instead we're all one emergency vehicle parked across the street away from a terrible accident.
The point of my comment is that depending on how you interpret the description "predicts the next token", something that can do this well enough should be superhumanly clever. The task is "AI Complete".
Don't misinterpret me as saying that GPT is superhumanly clever. I'm just saying that something that's sufficiently good at what people think GPT is doing, should be superhuman.
I gave ChatGPT 5 questions from a verbal reasoning test on sequence of words and it got 2/5 right.
Here is sample question:
Arrange the words given below in a meaningful sequence.
1. Poverty 2. Population 3. Death 4. Unemployment 5. Disease A) 2, 3, 4, 5, 1 B) 3, 4, 2, 5, 1 C) 2, 4, 1, 5, 3 D) 1, 2, 3, 4, 5
ChatGPT>
B) 3, 4, 2, 5, 1
Unemployment can lead to poverty, which can lead to disease. Disease can lead to death. The population of an area is affected by all of these factors. Therefore, the correct order is 3, 4, 2, 5, 1.
Correct answer was C
I would expect if it was effective at reasoning it should be strongest on verbal associations. And while it is possible that enhancements to future language models might provided them with better reasoning ability I don't believe you can simply push the existing design to that limit.
The poor results from the verbal reasoning tests demonstrate that reasoning is more complicated than what a transformer based language model is capable of.
Overall, the argument is moot because the statement made in the article was it was a reasoning engine which is not equivalent to being capable of reasoning.
"Machined" means a precise implementation. A machined reasoning engine would be precise, accurate, and consistent. GPT-4 is impressive autocomplete, but not what we would expect for machined reasoning.
"Next token prediction" taken to a certain mathematical limit implies reasoning.