Re:A visual proof that neural nets can compute any function
by YeGoblynQueenne on Monday March 07, 2022 @09:39AM
(#30588327)
They simultaneously have not enough information, and too much information. A human child can learn what a chicken is by seeing a chicken once, but "seeing a chicken" is not the same as the representation of a chicken as a set of pixels in an image. It's ... something else, that we don't quite have the words to describe. Clearly, when we look at things in the real world we take in much more information than when a CNN takes in an image. Machine learning datasets have too much data, but it's the wrong kind of data and they're the wrong kind of learning system, to learn what humans can learn from "single examples".
And humans can also learn about radically new categories of things that they have never experienced just by looking at an image, even a low-quality image.
And that's humans which are like the apex of intelligence on the planet (I know, right?). Neural nets can't even built the representation of the world that a simpler animal, like a cockroach, must have to be able to exist autonomously in its environment. We don't have anything like a machine that can get close to the ability of a cockroach for autonomy. It's mad. But people keep thinking, "hey, we'll just use more data and more layers" and get to the moon by jumping higher with a larger pogo stick.
And in a static photo of a bent stick, or a fluffy critter, there simply isn't any information to tell whether this is a bent stick or a stick in water; or whether it's a cat or a dog. The intelligent response is not "I don't know", and it's not "60% it's a cat, 40% it's a dog". It's "here is the set of actions that need to be taken to create more data to be able to settle the question".
And creating that set of actions is completely different from current approaches. No state of the art GAN can say "you need to view the subject from a steeper angle, check for a reflection to see if it's in water" or "poke it with a stick, see if it meows or barks", because they don't have enough information about the world in their train g sets to be able to even know that these are possibilities.