> AlphaGo didn't learn the rules of Go. It has no idea what those rules are, it's just been trained through trial and error not to break them. That's not the same thing. It's not approaching chess or Go as an intelligent thinking being, learning the rules and working out their consequences

There are indeed some people who learn chess by "reading the manual". Or learn a language by memorizing grammar rules. Or learn how to build a business by studying MBA business theories.

There are also tons of other people who do the opposite. They learn by simply doing and observing. I personally have no idea what an "adverb" is, but people seem perfectly happy with the way I write and communicate my thoughts. Would my English skills count as general intelligence, or am I just a pattern-recognition automaton? I won't dispute the pattern-recognition part, but I somehow don't feel like an automaton.

I can certainly see the potential upsides of learning some theory and reasoning from first principles. But that seems too high a bar for general intelligence. I would argue that the vast majority of human decisions and actions are made on the basis of pattern recognition, not reasoning from first principles.

One last note: "working out their consequences" sounds exactly like a lookahead decision tree

Alphago and it's kind are doing some things that we do, for sure. We do utilise pattern recognition and some of the neurological tools we bring to bear on these problems might look a bit like Alphago.

The thing is those are parts of our neurology that have little to do with general intelligence. I think it's becoming clear that they are merely cognitive tools we bring to bear in the service of our higher level general intelligence machinery. In that sense high level Go and Chess players turn themselves into single-task solvers. They're better at bringing that experience and capability to bear in other domains, because they have general intelligence with which to do so, but those specialised capabilities aren't what make them a being with general intelligence. Or if specialising systems are important to general intelligence, it's as just a part of a much broader and more sophisticated set of neurological systems.

I can't agree with you. I know many chess players at master and Grand Master level. Look at Bobby Fischer too. Human specialization does not carry over very well to other tasks, only marginally...

I don't think that's a disagreement, I think you're right. Most of the benefit someone like that would get from their competence in Chess or Go would be incidental ones. In fact I would say your experience confirms my understanding of this, optimizing for a single domain in the way Alphago does or even in the way humans do, has little to do with general intelligence.

Ah, I misread. So we agree then. On the other hand, I would not be surprised if the hippocampus was highly developed in chess players like Bobby Fischer which could translate into better spatial reasoning. Perhaps general intelligence is best trained by variance.... Not targeted training.

You could be right, and for the record I upvoted your comment for the contribution regarding you experience with high level chess players. I think the downvotes you’re getting are regrettable.

I appreciate your sentiment. This field is my focus right now. My bachelors is in BioChemMed but I am doing a master in CS and have finished many courses including the free ones by Hinton, LeCun, and Bengio.

Here is my strongest prediction:

AGI is only possible if the AGI is allowed to cause changes to its inputs.

Current ML needs to be grafted towards attention mechanisms and more boltzmann net / finite/infinite impulse response nets.

> "... if AGI is allowed to cause changes to its inputs."

Could you elaborate on this point ?

Do you mean that the AGI could change the source of inputs, or change the actual content of those inputs (e.g. filtering) or both?

And why do you think this is a critical piece ?

Both. Attention changes the source. Action interacts with the source, modifying it. But the environment will need to respond back. This is reminiscent of reinforcement training but is more traditional NN except where the input is dynamic and evolving with every batch not only in response to the agent but in response to differential equations or cellukar automata / some type of environment evolution. AGI should be able to change the environment in which it inhabits. Attention in some respects is a start - it is essentially equivalent to telling reality to move the page and watching it happen. Until we have attention AND data modification, we will keep getting the specialized NN we are used to.

But in this classification, isn't "general intelligence" just a meta-single-problem-solver, solving the problem of which single purpose solver to bring to bear on this task?

I think I think, but might I just be using a single problem solver that gives the appearance of thinking?

I suspect the way we think in terms of clear symbols and inference isn't actually how we think but a means of providing a post-hoc narrative to ourselves in a linguistic form.

Edit: Which kind of explains the failure of good-old fashioned symbolic AI as it was modelling the wrong thing.

That makes a lot of sense. An internal narrator on events explaining them to the passenger, rather than the driver of said events.

Definitely not my idea though - couldn't find any good references to where I read about that idea.

[NB I worked in good-old-fashioned AI for a number of years]

> AlphaGo didn't learn the rules of Go. It has no idea what those rules are, it's just been trained through trial and error not to break them.

When given a problem it has never seen before it was able to acquire knowledge of the problem and then apply that knowledge as to solve the problem. Thats the definition of learning and intelligence that can generally be applied to any problem.

> I think it's becoming clear that they are merely cognitive tools we bring to bear in the service of our higher level general intelligence machinery.

What are you on about? Cognition and intelligence are the same thing if it's capable of cognition or as you put it applying "cognitive tools" then it's capable of intelligence.

>When given a problem it has never seen before it was able to acquire knowledge of the problem and then apply that knowledge as to solve the problem. Thats the definition of learning and intelligence that can generally be applied to any problem.

It can't be applied to any problem though. Take the example i gave elsewhere of a game where you provide the rules, and as the game progresses the rules change. There are real games that work like this, generally card games where the cards contain the rules, so as more cards come into play the rules change. Alpha Zero cannot play such games, because there isn't even a way to provide it with the rules.

>> I think it's becoming clear that they are merely cognitive tools we bring to bear in the service of our higher level general intelligence machinery. > >What are you on about? Cognition and intelligence are the same thing if it's capable of cognition or as you put it applying "cognitive tools" then it's capable of intelligence.

I'm saying that human minds apply many cognitive tools, and that Alphago is like one of those tools. It's not like the part choosing and deploying those tools, which is the really interesting and smart part of the system.

The human brain consists of a whole plethora of different cognitive mechanisms. Cognition is a broad term of a huge variety of mechanisms, none of which by themselves constitute all of intelligence. A lot of people look at Alphago and say aha, that's intelligence because it does something we do. Yes, but it only does a tiny, specialist fragment of what we do, and not even one of the most interesting parts.

In a Platonic dialogue, they discuss the definition of knowledge as "true belief with an account". You have true beliefs about language, but you don't know it in the Platonic sense if you can't explain it to someone else. Another way I've heard this defined is, you don't know it if you couldn't write the algorithm for it.

By that definition, most people don't have "knowledge" over most things which they believe and act on. And yet, no one accuses them of not possessing "general intelligence".

If an AI shows the same capabilities as the average human being, I would say that is AGI by definition. Regardless of whether it meets the requirement for Platonic Knowledge.

Really most people learn by a mix of the two. It’s going to take you a lot longer to learn chess is you have no clue on the rules.

But on the other hand if you get into rote memorization before you start the game it’s going to slow you down by having no context.

I think people do both. You learn the rules in order to play a game or perform a task, but with practice you end up training a task specific system that "knows" how to do it without thinking about the rules and perhaps without knowing them.

I think if you have a framework that can produce arbitrary single-task solvers (which AlphaZero can't yet), you would have something indistinguishable from AGI, since communication between single-task solvers is also kinda just a single-task solver.

It's certainly not the most efficient way to use our current hardware, and it's not clear to me how big some of these neural nets would have to be, but if we had computers with a trillion times the memory capacity and speed, IMO it'd certainly work on some level.

How would a single-task solver, or hierarchy of them, go about constructing a conceptual model of a new problem domain? The problem with a solver is it only really goes in a single direction, but when modeling a system you spend a huge amount of time backtracking and eliminating elements that yielded progress at first but then proved to be obstacles to progress later. You also need to be able to rapidly adapt to changing requirements.

Imagine playing a game of Chess in which the pieces and rules gradually changed bit by bit until by the end of the game you were playing Go. That's much closer to what real life problems are like and a human child could absolutely do that. They might not be much good at it, but they could absolutely do it even without ever having played either game before just learning as they went. Note to AGI researchers, if your chatbot can't cope with that or a problem like it without any forewarning, don't bother applying for a Turning Test with me on the other side of the teletype.

they'd do it like we do: by comparing the new situation to previous ones we know about, and applying the model that fits best, and then adapting to results.

For humans, the more previous ones we know about, the better, because we have more chance of applying a model that works in the new environment. That's called "experience".

That’s a very broad, general description of behaviour that doesn’t actually describe an implementation. In fact it could apply to many completely different possible implementations. I suspect though that humans do more than this, that we have a way of either constructing entirely new models from scratch, or of dramatically adapting models to new situations without mere iterative fitting to feedback. Humans are actually capable of reasoning effectively about entirely new ideas, scenarios and problems. We have little to no idea how we do this.

I don't know. I'm not so sure that we can create new working models from scratch. We definitely learn by iterative feedback: babies wiggle stuff and watch what happens to learn how to move their bodies. Learning to ride a bicycle is mostly about falling off bicycles until you learn how not to.

I've seen people apply their normal behaviour to situations that have changed, and then get totally confused (and angry) as to why the result isn't the same. Observe anyone travelling in a new country for examples ("why don't they show the price with the sales tax included here? This is ridiculous!").

In a perfect world, sure, we'd construct a rational mental model of a new situation and test it carefully to ensure it matched reality before trusting it, and then apply it correctly to the new situation. But it's not a perfect world, and people don't actually do that. Usually we charge in and then cope with the results.

Of course, I'm not saying that AI should do that. It'll be interesting to see how a "good" general AI copes with a genuinely new situation.

I think we apply radically different cognitive machinery to physical skills like riding a bicycle, compared to playing a card game where the rules are on the cards, and you have no idea what rule will be on the next card or even what rules are possible. We can train Chimps to ride bicycles, so they have the cognitive machinery for that, but we can't teach them to play these kinds of card games.

Interesting. True. But is that because we lack the communication skills to explain the rules to chimps, or because they lack the cognitive modelling ability to understand those rules?

Seems to me you're just describing reinforcement learning. Youre just saying a human child can adapt to the new problem faster than the AI can adapt, which is true but not the argument you've been making in this thread.

It's not reinforcement learning, because the child can do it the first time, so there's no reinforcement. I have kids, so many times I have played games with them successfully, purely from a description of the rules and playing as we went. They even beat me once the first time we ever played a game, by employing a rule at the end which had never come up in previous play. Compared to that Alphago isn't even in the race, because we can't even tell it the rules.

> since communication between single-task solvers is also kinda just a single-task solver.

It would be nice if it worked like that but I think you're massively underestimating the problem set here. I'd suggest its more like the architectural glue one needs as an engineer writing a command line util and a fully fledged piece of Enterprise solution (i.e. orders of magnitude).

Of course because we don't actually know how intelligence exactly works we're both guessing here.

The only way i think it can be done is simulated evolution.. be that simulated evolution of neural nets or something else.

As others have mentioned here though.. this becomes horrifying if we've created something sentient to kill in games or enslave.

I've been thinking for a while that use of AI in games might become a civil rights frontier in about 30 to 50 years or so

The open ended simulations, similar to earth conditions, might general enough to sprout some artificial general intelligence. Put multiple intelligences in a massive multiplayer online world and have them compete for shelter and resources. It's an environment that we know has produced intelligence.

It may be a brutal struggle, but perhaps that struggle is important. Perhaps having a simulated tree fall on you is more meaningful than being reaped by some objective function at the end of an epoch.

I think you’re on a potentially productive path, but it took 2 billion years of evolution in a staggeringly vast environment like that to produce results. The question is really how to shortcut that process, but training environments may well have a role to play.

Reverse engineering a human mind would be another approach.

This is often overlooked but it's the only approach that is pretty much guaranteed to succeed given enough time. That said, it's also likely that AGI will come about way earlier from another approach (just as planes came before the "robot birds").

While I agree this leads to haunting outcomes. e.g. If we create a successful interface then whats the point of building our own digital pastiches if we can just strap in the real thing?

Check out OpenWorm. They're trying to reverse engineer the simplest organism with a nervous system, a nematode with 302 neurons. They're making progress, but not very fast. That approach is going to be a long haul.

Ted Chiang wrote an interesting novella, The Lifecycle of Software Objects, about that very subject

https://en.wikipedia.org/wiki/The_Lifecycle_of_Software_Obje...

Another one is Crystal Nights by Greg Egan. Full text here:

https://web.archive.org/web/20090604111226/http://ttapress.c...

Or Iain Banks' take on the subject, in Surface Detail(https://en.wikipedia.org/wiki/Surface_Detail)

edit: wrong book ;)

> simulated evolution

Isn't that what genetic algorithms are?

yeah, one kind i guess. Or perhaps they cover all kinds?

well, "kill" becomes moot if the code is preserved. Like "killing" another player in a multi-player game. You're not actually killing them.

This may be a clunky analogy, but is this fundamentally different than a killing a human as long as we keep a record of their DNA sequence? Maintaining the information doesn't seem enough to negate snuffing out the execution of that information

The generic AI could be "playing" in a thousand virtual environments at once. Killing one of them doesn't really have a parallel in human life, or ethics.

I mean, yes, you killed a sentient being. But if that sentient being has a thousand concurrent lives, then what does "killing" one of those lives even mean? And if it can respawn another identical life in a millisecond, does it even count as killing?

I suspect that having sentient virtual entities will provide philosophy and ethics majors a lot of deep thinking room. As it already has for SciFi authors.

Would this opinion change if scientists were able to prove the multiverse theory where there are infinite numbers of "us" living concurrent lives as well?

Not if we persist the state of the mind before we turn them off. Can't do that with humans

yet

> They are just very highly advanced single-task solvers.

What if AGI is just a combination of very highly advanced single-task solvers?

I happen to believe that it is an emergent behavior once the complexity gets high enough, so AGI might just be a (large?) collection of AlphaGo solvers connected to different inputs.

> It's like an image classifier that can identify an apple, but has no idea what an apple is

Kind of like humans then.

A child picks up an apple but doesn't know what an apple "is". It doesn't even have the vocabulary to describe it.

As adults we know what an apple is because we understand it as a concept, the ideal "apple", and can manipulate the concept into areas way outside the original concept (say, the phrase "apple of my eye").

The child does know that the apple is a thing though. That it’s a separate object that can be carried around. Computer vision ML systems don’t even know that!

All they know is how to recognize a common pattern on a pixel grid, after seeing a large number of examples, and then draw a box around it.

The fact that a child has a body and can manipulate the world with all 5 senses working in concert should not be underestimated.

A child comes pre-programmed to put things in their mouth. They also have very sophisticated reward functions built-in that identify tasty sugars entering their mouth.

Very quickly (assuming said child doesn't eat something too bad), in the absence of an external oracle, the child learns a very productive mental model of what an apple is.

This type of feedback loop seems eminently translatable to machine learning, assuming we can encode the concept space in a way that allows the model to be encoded and trained in a reasonable set of constraints

Right, but that’s actually just a tiny part of the puzzle. The cognitive machinery that knows about edibility, deco possibility (how objects can be decomposed into parts and have internal structure), compost leaves properties (how the parts of an Apple contribute to its attributes as a whole), it’s relationship and interactions with other objects in the environment. All of that cognitive architecture might be a target for your feedback loop, but isn’t a solver and won’t work like a solver.

Yes, but you did not manipulate the concept. You did not invent that phrase, but simply learned its meaning. A machine can do that.

Every child reinvents the concept. It wasn't there on birth, and the word and phrases didn't contain it. That's a bit difficult topic to wrap ones head around, but it is critically important to differ between signified (the concept) and signifier (the words etc.).

The child develops concepts and is able to create and evaluate inferences, and thus able to understand metaphors etc.

The concept is what most AI approaches lack. Googles image search can identify apples, and cherries, and probably can categorize both as fruits, but it can't infer that this probably contains seeds, is a living being etc.

Or even that it is a three dimensional object, or what that means.

You're the only person who mentioned metaphors here. My intuition tells me metaphors will be key to developing AGI. Metaphors literally generalize; they predict; they organize and catalogue. Formation, testing, and introspection of metaphors seems to be a way forward.

If you are interested in this direction of research: There is a big body of work regarding human cognitive processes and the role of metaphor. I would suggest "Philosophy in the flesh" by Lakoff and Johnson. A hefty work, but that was one of the publications that fundamentally changed my perspective on the human mind. The concept of embodied reasoning was eye-opening for me.

As I have an academic background in learning theory and developmental psychology, I'm pretty pessimistic about the current AI trend, autonomous driving etc. Most smart people in the field are chasing what are effectively more efficient regression functions for over 60 years now, and I almost never stumble upon approaches that have looked at what we know about actual human learning processes, development of the self etc.

Moravec's paradox[1] IMO should have been an inflection point for AI research. This is the level of problems AI research has to tackle if it ever wants to create AGI.

[1] https://en.wikipedia.org/wiki/Moravec%27s_paradox

Related: Martin Hilpert has an excellent lecture/video on metaphor, as part of his Cognitive Linguistics course. Well worth a watch if this is a topic that interests you.

https://www.youtube.com/watch?v=R0BYLpwSM6E