Re:Apocalypse soon: the scientists preparing for the end times
by TeMPOraL on Tuesday September 30, 2014 @08:13AM
(#8387726)
> I don't think AI will be "programmed" in a traditional sense of this word. AI will need to go through a learning process, where it will have a chance to learn all about humans, and the environment.
Given [0], I think that it's the only plausible way we can have to make it follow human values.
> True AI (tm) on the other hand will be smart enough to know that making paper clips out of humans is simply stupid.
That's exactly The Misconception the whole movement around Friendly AI is trying to debunk. See [1].
> I'm sorry but if the paper clip AI doomsday scenario is the best this institute could come up with to explain why AI could be dangerous then I'm not impressed.
Well, paperclip maximizer is just a toy example made up for explaining why general AIs are dangerous to those for whom this fact isn't obvious.
Given [0], I think that it's the only plausible way we can have to make it follow human values.
> True AI (tm) on the other hand will be smart enough to know that making paper clips out of humans is simply stupid.
That's exactly The Misconception the whole movement around Friendly AI is trying to debunk. See [1].
> I'm sorry but if the paper clip AI doomsday scenario is the best this institute could come up with to explain why AI could be dangerous then I'm not impressed.
Well, paperclip maximizer is just a toy example made up for explaining why general AIs are dangerous to those for whom this fact isn't obvious.
[0] - http://lesswrong.com/lw/ld/the_hidden_complexity_of_wishes/
[1] - http://wiki.lesswrong.com/wiki/Orthogonality_thesis