Re:Is there a simple algorithm for intelligence?
by Strilanc on Friday July 10, 2015 @11:49AM
(#9864615)
AIXI solves a useful class of problems, but definitely does not solve all problems [1].
The main class of problems it fails to solve are ones where it has to model itself as an agent in the world.
For example, AIXI is a bit accidentally-suicidal. Its world models have no term for "I am here"; e.g. it will fail to realize that disconnecting the power to save money would stop the inference process instead of informing it. It needs many many examples of dying in order to learn to avoid it, but you only get one chance. (Evolution worked around this problem by making copies, and preferentially keeping copies that were less accidentally suicidal.)
The main class of problems it fails to solve are ones where it has to model itself as an agent in the world.
For example, AIXI is a bit accidentally-suicidal. Its world models have no term for "I am here"; e.g. it will fail to realize that disconnecting the power to save money would stop the inference process instead of informing it. It needs many many examples of dying in order to learn to avoid it, but you only get one chance. (Evolution worked around this problem by making copies, and preferentially keeping copies that were less accidentally suicidal.)
1: http://lesswrong.com/lw/jg1/solomonoff_cartesianism/