Re:Past Performance is Not Indicative of Future Results (2020)
by js8 on Sunday August 01, 2021 @05:53AM
(#28025124)
> I share the skepticism towards any progress towards 'general AI' - I don't think that we're remotely close or even on the right path in any way.
I actually think that AGI is deceptively simple. I don't have a proof, but I have a (rather embryonic, frankly) theory of how is it gonna work.
I believe AGI is an analogue of third Futamura projection, but for (reinforcement) learners and not compilers.
So the first level is you have problem and a learner, and you teach learner to solve the problem. The representation of the problem is implicit in the learner.
The second level is that you have a language, which can describe the problem and its solution, and a (2nd level) learner, and you teach the 2nd level learner to create (1st level) solvers of the problem based on the problem description language. The ability to interpret the problem description language is implicit in the 2nd level learner.
The third level is, you have a general description language that is capable of describing any problem description language, and you teach the 3rd level learner to take a description of the problem description language, and produce 2nd level learners that can use this language to solve problems created in it.
Now, just like in Futamura projections, this is where it stops. You have a "generally intelligent" creature on the 3rd level. You can talk to them on level of how to effectively describe or solve problems (create a specialized language for it) and they will come all the way down with the way to attack (solve) them.
In humans, the 3rd level, general intelligence (AKA "sentience"), evolved eventually from the 2nd level, and it was a creation of the general internal language (which probably co-evolved to be shared). The 2nd level is an internal representation of the world that can be manipulated, but only ever refer to the external world, not itself, so it allows creatures to make conscious plans, but lack the ability to reflect on the planning (and also learning) process itself. The "bicameral mind" is a theory how we acquired 3rd level from the 2nd, and the 3rd level is why "we are strange loops".
Anway, the problem is, the higher you go up the chain, the harder it becomes to create the learner, it's a lot more general problem. But I think the ladder must be, and should be, climbed. I believe that Deepmind (and RL research) has solved the 1st level, is now working on the 2nd level, but they already somewhat dimly see the 3rd level.
I actually think that AGI is deceptively simple. I don't have a proof, but I have a (rather embryonic, frankly) theory of how is it gonna work.
I believe AGI is an analogue of third Futamura projection, but for (reinforcement) learners and not compilers.
So the first level is you have problem and a learner, and you teach learner to solve the problem. The representation of the problem is implicit in the learner.
The second level is that you have a language, which can describe the problem and its solution, and a (2nd level) learner, and you teach the 2nd level learner to create (1st level) solvers of the problem based on the problem description language. The ability to interpret the problem description language is implicit in the 2nd level learner.
The third level is, you have a general description language that is capable of describing any problem description language, and you teach the 3rd level learner to take a description of the problem description language, and produce 2nd level learners that can use this language to solve problems created in it.
Now, just like in Futamura projections, this is where it stops. You have a "generally intelligent" creature on the 3rd level. You can talk to them on level of how to effectively describe or solve problems (create a specialized language for it) and they will come all the way down with the way to attack (solve) them.
In humans, the 3rd level, general intelligence (AKA "sentience"), evolved eventually from the 2nd level, and it was a creation of the general internal language (which probably co-evolved to be shared). The 2nd level is an internal representation of the world that can be manipulated, but only ever refer to the external world, not itself, so it allows creatures to make conscious plans, but lack the ability to reflect on the planning (and also learning) process itself. The "bicameral mind" is a theory how we acquired 3rd level from the 2nd, and the 3rd level is why "we are strange loops".
Anway, the problem is, the higher you go up the chain, the harder it becomes to create the learner, it's a lot more general problem. But I think the ladder must be, and should be, climbed. I believe that Deepmind (and RL research) has solved the 1st level, is now working on the 2nd level, but they already somewhat dimly see the 3rd level.