Logically thinking the only way that ChatGPT could do well on a test of reasoning without engaging in reasoning is if the test is useless as a test of ability to reason.
Since humans use this test to select people with good reasoning skills, why do you think it is an inappropriate test for a robot?
The test is not useless, the test's use case is to create a standardised datapoint to pick people to limited spots. That' what the test does, you need to attach false attributes to the test(like measures intelligence) to associate these attributes to the machine and this is a fallacy because although the test probably says something about the intelligence of the human, it does it by measuring meta attributes of the intelligence of the human(like speed of solving problems that the human was supposed to train on).
This is also why there is the "Goodhart's law" phenomenon where in this case students instead of learning physics might start putting all their effort on learning to solve physics tests and you end up with students with very high scores in physics but with no understanding of physics.
The reason I prefaced my opinion with "I've used ChatGPT a lot" is to show that this is my personal impression based on my real everyday experience - I'm not guessing or deducing. I personally interact with ChatGPT in a way that satisfies me as to the fact that it is an intelligent machine capable of very usefully handling abstract notions.
ChatGPT has achieved 80% in this user's run, will refuse to play at all the next run, will get 45% the following run etc.
Stochastic randomisation makes the chatbot useful and provides these moments of impressive capability, it also means unlike humans that capability is in no way consistent.
> ChatGPT has achieved 80% in this user's run, will refuse to play at all the next run, will get 45% the following run etc.
While I agree with your conclusion, I want to slightly modify the first sentence. I know this is just one anecdote but I took the SAT 1 twice and did somewhat better the second time. I think this is common as you are more familiar with the test the second time.
My SAT score ranged from 980 to 1450 with 4 tries, increasing each time and had there been a reason or benefit I'm certain I could have eventually gotten a perfect score. I don't see how anyone could not get better over time.
It's trained on the human data that was chosen as more impressive, and trained to pull the most impressive parts out of that data.
Fundamentally, while GPT may be "learned", it is not "logical". Instead of learning potential logical conclusions, it has learned potential semantic responses. It doesn't match the meaning of words to other meanings: it matchesa set of expressions to other expressions.
The impressive part is that the logical conclusions already present in the training data get presented by GPT in a semantically coherent way.
Sure, we can keep pretending this I guess.
That's a bit condescending.
ChatGPT isn't remotely close to "general creative intelligence, problem-solving skills and ability to manipulate abstract thoughts daily". It only pretends, in a pretty impressive way. A bit like you can train your dog to play dead when you point your finger at them and say "bang", but you should not conclude from it that your dog understands the concept of death.
> Since humans use this test to select people with good reasoning skills, why do you think it is an inappropriate test for a robot?
Well it is already not a perfect solution for humans. When you use a metric to rate people, then they start optimizing for this metric. Typical example is academic rankings: if you are ranked based on how many papers you publish, you will publish a lot. Doesn't mean you will do good research.
Now even if SAT was a good way to rank humans, that does not say at all it would be a good way for robots. If you test whether a cat is stupid or not by asking it to drive a car, then obviously it will be stupid. Asking ChatGPT to pas the SAT tests is probably like asking a student to pass it with infinite time and an internet connection. I guess that's not how SAT exams are usually done, right? (I am not in the US).
What are the kinds of things you are finding ChatGPT isn't able to understand?
Say you ask DeepL to translate a sentence about biology from Spanish to English, and it does it perfectly. Would you say that DeepL understands biology? Or even that it knows the languages like a human does? Or would you rather think that it has a huge database of texts in both languages, and it "just" built its translation based on similar sentences there (somehow cleverly interpolating between its data points)?
Because it "looks like it does" does not mean that "it does". That's how magicians trick you into pretending that they do magic. And that was my point about the dog playing dead. That does not mean that the dog understands that you were pretending to hold a gun, or that it understands that it plays "dead". The dog just "lies on the back" when you point your hand and say "bang", that's it.
> What are the kinds of things you are finding ChatGPT isn't able to understand?
Let me put it this way. Wouldn't you expect from an agent able to show "general creative intelligence, problem-solving skills and ability to manipulate abstract thoughts daily" that it could conduct research on its own? Why can't you say "please solve this mathematical problem: you have the universal knowledge in your database and a lot of compute power. Please send me a summary of your progress every day until it is solved". Or do you actually think that it can do that?
Yes, you can say "please solve this mathematical problem" and sometimes get a novel working solution that has never been published but is formulated on the spot. Rather than just output text, you can direct its behavior in profound ways, including successfully completing miracle, literally impossible tasks. That's right: ChatGPT can perform a miracle and successfully in one try complete a task that is by definition impossible, such that it breaks the very test itself.
You don't get that from a mere search engine.
That's where you get your miracles from.
I guess I won't convince you that ChatGPT does not manipulate abstract thoughts, but just words. It really just interpolates between texts it has read, sometimes resulting in interesting answers, sometimes resulting in complete bullshit. And it has absolutely no clue about how wrong its output is. Because it does not "think" or "understand". It really just brilliantly copy-pastes words that seem relevant.
My experience is that ChatGPT is not particularly good at reasoning, so I assume that either the test isn’t a particularly good test of reasoning or the result was a fluke.
It’s a text model, not a reasoning model, so I wouldn’t expect reasoning ability in the first place.
A human and a robot are both good at taking a bad test. This doesn’t say much about the robot.
Your body was able to transform what was in the soda (sugary water) into a completely different thing (ammonia and water), while preserving a key attribute (water).
Our assertion that the preserved attribute is "key" is the act of us choosing what to be impressed about.
GPT was trained to be impressive by preserving the logical soundness of expression when semantically transforming expression. It did not explicitly choose the logical conclusions: only the semantic match that contains them.
Expressing logical conclusions in natural language is inherently ambiguous, which makes this a very large and incoherent problem space.
Cola making the test positive doesn't mean that the test is fake or that the Cola has virus and ChatGPT scoring something on SAT doesn't mean it's equivalent to a person scoring the same and should study in Stanford.
These tests are designed to pick people to limited number of spots and doesn't say much more than the persons' preparedness for the test.