There is: Death, without reproducing. If an organism dies quickly but left children, they were biologically successful. If you didn't, then your evolutionary branch goes away.

I'm not sure this can be simulated, but it might be worth trying. Biological brains are the product of a billion years of the above strategy, so hopefully it's feasible to do the same thing.

Don't people already do that? Well, no. The strategy implies some sort of physics simulation with general rules, not a specific ruleset tuned to your domain.

> There is: Death, without reproducing.

The problem is that it's not describing the truth at every moment of life, just the total approximate fitness of that lifeform by the end of its lifetime. It would be time consuming and cost-inefficient to simulate many possible configurations of actors until the end of their lifetimes.

Although I do tend to think that given enough time (some millions of years), other intelligence will evolve on earth, from parrots or elephants or dolphins or octipuses or something. Though there's no evidence of a brain growth spurt that led to us, in other species; and it's not settled why we had it.

edit: This comment isn't a joke, I mean this seriously. Modeling a system after ourselves may not be the best idea if we're not the best possible system.

We're biased towards individualism and the illusion of personal intelligence making clever decisions using its own resources, but in fact our real intelligence is collective.

Individually, most humans are a little bit smarter than other animals, but not much.

Humans are also excellent mimics. We learn by copying, and a lot of what we consider intelligence is copied behaviour and copied beliefs - often applied unconsciously.

So we've created systems - spoken language, writing, science, technology - that can externalise, collect, and preserve the best of our learning, so the species as a whole, including average humans, can get the benefit of it.

This works fine as far as it does, but our political and economic systems are still founded on animal logic (i.e. personal advantage no matter the cumulative long-term cost) instead of conscious collective intelligence amplification and species optimisation.

Based on this, AI won't get far unless someone invents an analogous way to externalise and abstract learning, so different AI projects can bootstrap each other, and there are common systems for abstraction and generalisation.

Unsupervised learning isn't going to be all that interesting unless it can do that. It'll be a fun toy for certain classes of problems, but it won't be anywhere close to an AGI.

Long before language, babies spend hours every day just interacting with the physical environment, which is "true": the toy is here, not there. The body weighs a certain amount. If you move the head this way, you'll roll over. If you move it that way, no rolling. Etc.

And parents provide a different kind of guidance. We react differently to different sounds, actions, movements of child. Even if these are not "truth" philosophically, they represent for the child emotional truth.

I'm just a guy who reads things about AI on the Internet, but I have to say it sometimes seems to me like AI folks have funny ideas of what "intelligence" is, and how it develops. The description of "unsupervised" learning above doesn't comport with any human experience I recognize. IME, humans receive constant immediate feedback on everything we do in life.

I think the origin is communication and memory. When a being (X) signals to another being (Y) the origins of its pain (signal of increased death probability) and X and Y can remember this cause, they can begin to teach.

If the output of your networks performs well enough when it confronts the real world, you get to reproduce before you die, and some of the essence of your network gets to live for another generation.

If not, you die, and your network doesn't get to contribute to the next generation.

Is it possible that your conscious self is the oracle that provides feedback to subconscious networks?

I suppose an animal may have a need - it feel hungry. If it can satisfy that need then it should reinforce whatever it did to get not-hungry. I suppose that doesn't require conscious awareness. But at some level of complexity we have to be able to plan more complex actions (hunting, tracking) and be able to self-evaluate performance on the sub-tasks used to get food.

>>Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life

This is false for most of learning of human intelligence. Most of schooled human intelligence looks like supervised learning.

Innate intelligence and other animal intelligence looks like pre-trained models and some reinforcement learning.

I think you are severely underestimating humans, and probably animals too.

People have multiple modes of learning. You can try to get an A in school ("optimize" for the objective function), but you also might be skeptical of what the teacher tells you. Likewise you might be skeptical of what you read in a book.

You can also correct the teacher based on common sense and what you already know.

Currently, a computer can do no such thing. I think he's right that there is a fundamental difference.

Humans also learn from extremely small amounts of data. (Sometimes they learn badly, but they can change their minds, or not if they still manage to navigate the world.)

What does this mean? That 18 years of schooling is a "small amount of data"? Do you think you show 2 pictures of an airplane to a child and they know what an airplane is? Do you even have kids?

You appear to be thinking of just facts and figures. AI is not that.

5-year old humans have picked up a lot. For instance, I knew about the dangers that hot surfaces, sharp objects, angry cats, and big sisters represented.

I had learned to walk, run, swim, ride a bicycle, and catch a ball.

The list is almost endless. In all of those cases, if there is any supervision it is minimal.

Starting to suck when they feel something on their lips _is_ a reflex though, it isn't learnt.

Clearly there is stuff pre-programmed in the DNA and some animals e.g. are able to hold balance right after birth. But apparently most of this stuff in case of humans is instructions how to build a learning system, rather then encoding the complexity of the world in a strand of DNA. I grant you, the evolution did not have the time to encode ways to handle lego bricks and yet 2yr olds spontaneously acquire ability to manipulate these objects way beyond what any contemporary robot can do.

[1] https://en.m.wikipedia.org/wiki/Primitive_reflexes

I'd argue that we see clear examples of such in animals that do require those for immediate survival. We ought to think about it as a consequence of natural selection rather than a 'necessity' that is somehow intelligently pre-programmed for survival. It's far more likely that path to early survival traits were mutational (accidental) rather than supervised.

Children learn the rules of grammar long before they realize there are any rules at all. Tenses and plurals, etc.

You would be surprised how much infants learn from listening around them and how long it takes to learn that language, and how a brain interprets language was formed through so many years of evolution to be highly tuned to learn language fast.

I understood your previous comments as diminishing the sheer amount of unsupervised learning we do as humans.