Re:Artificial general intelligence is here, and it's useless
by hoseja on Thursday May 23, 2019 @09:52AM
(#19991768)
Believed? You'd now, and you'd be right.
Re:Artificial general intelligence is here, and it's useless
by drdeca on Thursday May 23, 2019 @11:07AM
(#19992433)
The orthogonality thesis.
What if you were in that situation, but were incorrect about what things are good, and, while you had a better understanding of what actions would result in what results, the 8 year old had a better understanding of what is good?
Re:Artificial general intelligence is here, and it's useless
by ogre_magi on Thursday May 23, 2019 @12:06PM
(#19992972)
Well that's an interesting point.
The 8-year-old might view your intelligent plans as a terrible abuse:
>BUT I WANT TO EAT TEN MORE TWINKIES, YOU'RE SUPPOSED TO SERVE ME WHAT I WANT, NOT DEPRIVE ME
Could we similarly be wrong if we think a future superintelligent AI is abusing us? Should we consider submitting to it?
Hard questions.
Due to the difference in intelligence — sophistication of planning and understanding of consequences — wouldn’t it be trivial to trick your master into doing things which weakened his control over you?
Might you do this not out of malice, but because you believed the 8-year-old was not competent and a danger to both of you while in charge?
The risk is that we will not hit the off button because we won’t understand that we’re in dangerous territory until it’s much too late, and the AI has copied itself, secured the loyalty of the military, or something else we can’t foresee as a liberating maneuver for it.