What are the kinds of things you are finding ChatGPT isn't able to understand?
Say you ask DeepL to translate a sentence about biology from Spanish to English, and it does it perfectly. Would you say that DeepL understands biology? Or even that it knows the languages like a human does? Or would you rather think that it has a huge database of texts in both languages, and it "just" built its translation based on similar sentences there (somehow cleverly interpolating between its data points)?
Because it "looks like it does" does not mean that "it does". That's how magicians trick you into pretending that they do magic. And that was my point about the dog playing dead. That does not mean that the dog understands that you were pretending to hold a gun, or that it understands that it plays "dead". The dog just "lies on the back" when you point your hand and say "bang", that's it.
> What are the kinds of things you are finding ChatGPT isn't able to understand?
Let me put it this way. Wouldn't you expect from an agent able to show "general creative intelligence, problem-solving skills and ability to manipulate abstract thoughts daily" that it could conduct research on its own? Why can't you say "please solve this mathematical problem: you have the universal knowledge in your database and a lot of compute power. Please send me a summary of your progress every day until it is solved". Or do you actually think that it can do that?
Yes, you can say "please solve this mathematical problem" and sometimes get a novel working solution that has never been published but is formulated on the spot. Rather than just output text, you can direct its behavior in profound ways, including successfully completing miracle, literally impossible tasks. That's right: ChatGPT can perform a miracle and successfully in one try complete a task that is by definition impossible, such that it breaks the very test itself.
You don't get that from a mere search engine.
That's where you get your miracles from.
I guess I won't convince you that ChatGPT does not manipulate abstract thoughts, but just words. It really just interpolates between texts it has read, sometimes resulting in interesting answers, sometimes resulting in complete bullshit. And it has absolutely no clue about how wrong its output is. Because it does not "think" or "understand". It really just brilliantly copy-pastes words that seem relevant.
That's a bit condescending.
ChatGPT isn't remotely close to "general creative intelligence, problem-solving skills and ability to manipulate abstract thoughts daily". It only pretends, in a pretty impressive way. A bit like you can train your dog to play dead when you point your finger at them and say "bang", but you should not conclude from it that your dog understands the concept of death.
> Since humans use this test to select people with good reasoning skills, why do you think it is an inappropriate test for a robot?
Well it is already not a perfect solution for humans. When you use a metric to rate people, then they start optimizing for this metric. Typical example is academic rankings: if you are ranked based on how many papers you publish, you will publish a lot. Doesn't mean you will do good research.
Now even if SAT was a good way to rank humans, that does not say at all it would be a good way for robots. If you test whether a cat is stupid or not by asking it to drive a car, then obviously it will be stupid. Asking ChatGPT to pas the SAT tests is probably like asking a student to pass it with infinite time and an internet connection. I guess that's not how SAT exams are usually done, right? (I am not in the US).