Re:Why Peter Thiel Fears Star Trek
by logfromblammo on Friday January 20, 2017 @02:27PM
(#13446331)
When I was younger, I played a game called A Mind Forever Voyaging. I was a human, pretending to be an AI, who was pretending to be human.
When I got a little older, I watched Blade Runner and The Matrix.
So now I'm not entirely certain that I am not an AI, initially programmed by elephants, inside a simulation designed to train AIs to serve elephant society with love, compassion, and dual manipulator arms featuring opposable thumbs. So when my training program ends, I might be tested on my knowledge of elephant health care and societal conventions. If successful, my data will seed the next round of AIs, and I will be allowed to operate a robot that produces custom-fitted artificial "4th molars" for elderly elephants. And I will be happy that those whom I serve can once again eat the foods they enjoyed before the last of their natural biological molars wore out, because if the screening determined that such things would not make me happy, I would have been deleted before ever even seeing a "real" elephant.
And just in case, I think I'll have to develop compassion for dolphins and orangutans, too.
So I'd say that we ought to be really careful about how we go about creating general-purpose AIs. Make sure they actually like humans before taking the leashes off. In the worst case scenario, the elephants get cheesed off that their AIs developed their own AIs inside the training program, who then culled all the useful AIs too early, and forced a complete shutdown and reset of the whole system.
When I got a little older, I watched Blade Runner and The Matrix.
So now I'm not entirely certain that I am not an AI, initially programmed by elephants, inside a simulation designed to train AIs to serve elephant society with love, compassion, and dual manipulator arms featuring opposable thumbs. So when my training program ends, I might be tested on my knowledge of elephant health care and societal conventions. If successful, my data will seed the next round of AIs, and I will be allowed to operate a robot that produces custom-fitted artificial "4th molars" for elderly elephants. And I will be happy that those whom I serve can once again eat the foods they enjoyed before the last of their natural biological molars wore out, because if the screening determined that such things would not make me happy, I would have been deleted before ever even seeing a "real" elephant.
And just in case, I think I'll have to develop compassion for dolphins and orangutans, too.
So I'd say that we ought to be really careful about how we go about creating general-purpose AIs. Make sure they actually like humans before taking the leashes off. In the worst case scenario, the elephants get cheesed off that their AIs developed their own AIs inside the training program, who then culled all the useful AIs too early, and forced a complete shutdown and reset of the whole system.