Re:Artificial intelligence: Our final invention?
by mindcrime on Wednesday December 18, 2013 @01:35PM
(#6929520)
3. Be wary of people trying to sell you ideas based on fear. It's usually snake oil, concealing some other agenda - such as trying to obtain or maintain grant money for projects.
That's very true, but I don't think it completely obliterates the relevance of the point being made in TFA. The truth is, we don't know exactly what a trans-human AI will be capable of, and whether or not it would do (or attempt to do) things that would be harmful to humanity. Now I'm certainly not arguing for stopping AI research! But maintaining awareness of this issue and reminding people to keep the unintended consequences in mind, is probably a valuable thing.
Don't get me wrong... I don't put a lot of stock in "fear of AI" (Matrix and Terminator jokes aside), so long as the AIs are locked in "the box" (to use Yudkowsky's term) and one can simply power the machine off and end the AI. Despite the (seeming) fact that Yudkowsky has talked his way out of "the box" a few times, playing the role of the AI, I think that we're a LONG way from an AI that can "go autonomous", talk it's way out of "the box" and become destructive. The scenario strikes me as highly unlikely, but just on the borderline of plausible enough to keep an eye on.
That's very true, but I don't think it completely obliterates the relevance of the point being made in TFA. The truth is, we don't know exactly what a trans-human AI will be capable of, and whether or not it would do (or attempt to do) things that would be harmful to humanity. Now I'm certainly not arguing for stopping AI research! But maintaining awareness of this issue and reminding people to keep the unintended consequences in mind, is probably a valuable thing.
Don't get me wrong... I don't put a lot of stock in "fear of AI" (Matrix and Terminator jokes aside), so long as the AIs are locked in "the box" (to use Yudkowsky's term) and one can simply power the machine off and end the AI. Despite the (seeming) fact that Yudkowsky has talked his way out of "the box" a few times, playing the role of the AI, I think that we're a LONG way from an AI that can "go autonomous", talk it's way out of "the box" and become destructive. The scenario strikes me as highly unlikely, but just on the borderline of plausible enough to keep an eye on.