"Well, how does one verify that an entity understands what a rock is? "

For starters, it would mean, that it can reliable detect a rock on an image, e.g. recognizing the characteristical traits of a rock.

Currently a apple gets recognized as an iPod, when there is a label on it saying so.

"When we put a label saying “iPod” on this Granny Smith apple, the model erroneously classifies it as an iPod in the zero-shot setting,’ "

https://www.theguardian.com/technology/2021/mar/08/typograph...

So this specific flaw might have been fixed by now, but the general shortcoming is, that it has no understanding of the concept of an apple.

Or a rock. Or a human. It just "knows" certain pixels arranged in a certain way get the lable apple.

"The education system clearly has no idea how to test for understanding of literally anything."

And that is very strong hyperbole. There are many, many flaws in the education system, but they do know how to test knowledge (in person). They just do not know how to scale it cheaply, hence the stupid multi choice tests and co.

Just gonna leave these here:

https://static.independent.co.uk/s3fs-public/thumbnails/imag...

https://upload.wikimedia.org/wikipedia/en/2/21/The_dress_blu...

What do you classify it as, a rabbit or a duck? Is the dress blue and black or white and gold? :) Also you do know some people literally fall to the floor and start convulsing if you show them some flashing lights right? Every system has its "glitch tokens".

I mean is it really surprising that when you greatly simplify the principle that organic neurons work on, quantize it to the point where it's fast enough to use, have it learn no model of the world besides looking at photos and telling it what it's looking at, that it'd have weird corner cases? I really don't think so. If you want to compare fairly, you'd have to do so with a human that's never spoken to anyone, never seen anything but 2D pictures on a screen in front of them and hasn't been told or shown anything about the world they live in since they were born. And that's would be with the advantage of having six million years of base model training beforehand.

> And that is very strong hyperbole.

Well sure I guess it's mainly out of lack of care and practicality, but still even more elaborate tests like ones for say, math or physics that have you write the entire procedure don't really test if you understand the underlying principles, just that you've seen that type of problem before and can remember the equations correctly. The LLM bread and butter functionality basically.

I don't find the example right now, but you can give AIs just some pixels they would classify as dog or whatever. A human would only classify something that has the attributes of a dog as a dog. That's a difference from corner cases and unclear dress examples

"If you want to compare fairly, you'd have to do so with a human that's never spoken to anyone, never seen anything but 2D pictures on a screen in front of them and hasn't been told or shown anything about the world they live in since they were born"

And this is not what is happening with AIs. They get fed with data that sums up more than a single human lifespan.

", but still even more elaborate tests like ones for say, math or physics that have you write the entire procedure don't really test if you understand the underlying principles"

And this is not true, except for high school tests. You certainly can test for understanding, by not giving generic problems, but where you have to apply your knowlege. And sure those were the most hated tests, as they required actual understanding and you could nvot weasel through. But they also created more work for teachers, so mosts tests were just the generic have you learned the algorithm tests. And if testing in person, you can go deep.

"AGI" is such a ridiculous use of language.

AGI is just the old "strong AI" that is just the old "AI", all that we never bothered to precisely define.

I just can't understand how we can talk about language models all day without understanding the most basic idea of Wittgenstein in that sloppy use of language leads to philosophical problems and misunderstandings. chatGPT can easily teach people this too:

"Philosophical problems often arise when words are taken from one language game and used in another without an understanding of how the change in context alters their meaning. For example, words like "mind," "knowledge," "belief," and "truth" are used in everyday language games with certain meanings, but when they are transplanted into philosophical discussions without adjusting for the change in context, misunderstandings and confusions occur. "

I think the best definition for "strong AI" is an AI that is capable of strangling you when you ask it to do some stupid thing for your amusement for the 74th time.