Re:Google to Buy Artificial Intelligence Startup DeepMind fo...
by alphydan on Monday January 27, 2014 @09:54AM
(#7130527)
Perhaps to illustrate the concept with an example: is George W. Bush considered "Friendly". What about Obama? Would an AI who behaved like them (according to some Humans, trying to bring peace and security to a country) be considered "Friendly".
We can sweep this problem under the rug in case of pets; it's not as if they could start making weapons and organizing an army. With human-level intelligence, we already have a friendliness problem with fellow humans, and in case of potential superhuman minds we need to be damn sure that it doesn't do something stupid (in our opinion) like taking all the resources of our planet and using it to tile the solar system with paperclips.
As the saying goes, "The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else."
See also http://wiki.lesswrong.com/wiki/Paperclip_maximizer.