Re:The darker side of machine learning
by goatlover on Friday October 28, 2016 @04:03AM
(#12813899)
"The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else." ~ Eliezer Yudkowsky
Skynet scenarios aren't what we should be worried about. It's the possibility of powerful self-improving AI systems which don't share our values because we didn't think things through properly in designing them. It may seem like they behave appropriately when constrained, but once they become smart enough to overcome constraints that apply to human beings, they could act in unanticipated (and undesirable) ways that we didn't consider. That's the challenge of designing a friendly AGI.
Skynet scenarios aren't what we should be worried about. It's the possibility of powerful self-improving AI systems which don't share our values because we didn't think things through properly in designing them. It may seem like they behave appropriately when constrained, but once they become smart enough to overcome constraints that apply to human beings, they could act in unanticipated (and undesirable) ways that we didn't consider. That's the challenge of designing a friendly AGI.