This is the debate, isn’t it? I think if we create tests for understanding and deliver them to people, we will find variations in what people understand. I think we will find the same for chatGPT.

But I suspect your notion of understanding is not measurable, is it? For you, chatGPT lacks something essential such that it is incapable of understanding, no matter the test. Or do you have a way to measure this without appeal to consciousness or essentialism?

Well, consciousness is part of the question, isn't it? We know that we are conscious (even if we can't precisely define what that means). Is ChatGPT conscious? I'm pretty sure the answer is no, but how do you prove it?

Does understanding require consciousness? Maybe yes, for the kind of understanding I'm thinking of, but I'm not certain of that.

How do you measure understanding? You step a bit outside the training set, and see if whoever (or whatever) being tested can apply what it has learned in that somewhat novel situation. That's hard when ChatGPT has been trained on the entire internet. But to the degree we can test it, ChatGPT often falls down horribly. (It even falls down on things that should be within its training set.) So we conclude that it doesn't actually understand.

Another part of all this is chatgpt was trained on the entire internet and still does a mediocre job when its doing well. That's an amazing amount of resources required to do all that to arrive at it being able to do what it does writing codes or whatever when a person typically requires a couple slices of pizza and general access to the internet which that person has Read almost none of.

How come humans are so efficient when these computers are using enormous amounts of energy? When will we have a computer that only requires a few slices of pizza worth of energy to get to the next step?

Give an example where ChatGPT4 falls down horribly on a novel concept where human would do better?

I disagree. If you play around with ChatGPT4 enough you can see it understands the concept. That is, it's able to model it and draw inferences from the model in a way that's impossible through just words and their relationships. "Sparks of AGI" paper gives some good examples, for instance where it's asked to balance a book, 9 eggs, a laptop, a bottle and a nail. I recently asked it to design a plan for a bathroom. It wrote and SVG mockup and got most of details right. For instance, it understood that sink and tub required both cold and hot water lines but toilet only requires a cold water line. These things are not possible with just words, you can see it's able to create an underlying world model.

> These things are not possible with just words

I don't think this is the case at all. Language is how we encode and communicate ideas/concepts/practicalities; with sufficient data, the links are extractable just from the text.

I don't see how two examples I gave are possible with just text. They require understanding spatial relationships between objects and their physical properties.

Our own understanding of spatial reasoning is tied in many respects to our hand-eye coordination, muscle memory and other senses: we learn to conceptualize "balance" by observing and feeling falling and when things are "about to" fall.

What GPT does is not "text" - although it centers that as the interface - but "symbols". The billions of parameters express different syntaxes and how they relate to each other. That's why GPT can translate languages and explain things using different words or as different personas.

So when we ask it to solve a spatial problem, we aren't getting a result based on muscle memory and visual estimation like "oh, it's about 1/3rd of the way down the number line". Instead, GPT has devised some internal syntax that frames a spatial problem in symbolic terms. It doesn't use words as we know them to achieve the solution, but has grasped some deeper underlying symbolic pattern in how we talk about a subject like a physics problem.

And this often works! But it also accounts for why its mathematical reasoning is limited in seemingly elementary ways and it quickly deviates into an illogical solution, because it is drawing on an alien means of "intuiting" answers.

We can definitely call it intelligent in some ways, but not in the same ways we are.