News for hackers. Stuff that matters.
Re:The A.I. Anxiety by sawwit on Tuesday December 29, 2015 @04:45PM (#10809284)
> "don't turn humans into paperclips" is part of the context of "make more paperclips"

The idea in this thought experiment is not that the AI parses this sentence in the context of human culture (then it would likely comprehend that the actual intention is to maximize the economic success and eventually the human preferences of its creator). What is meant is that the objective is crudely implanted into the AI as an ultimate goal, in a similar way to how sustenance, pain avoidance and affiliation are very basic goals in our cognitive system. It is not entirely obvious that this is a stupid thing to do; hence the thought experiment. Will the AI suppress its urge once it comprehends human culture enough to understand the intentions of its creator? Will it rather successfully learn all the tricks to convert matter into paperclips before it considers studying human values? If the AI does not have a curiosity objective, it will likely not care about us very much, apart from the information that helps it optimizing its objective function.

Re:The A.I. Anxiety by argonaut on Wednesday December 30, 2015 @03:33AM (#10811314)
I think it is way more likely we have AI that understands human context and values before we have AI that is super intelligent.
Re:The A.I. Anxiety by sawwit on Wednesday December 30, 2015 @06:30AM (#10811676)
What makes you so certain?
Re:The A.I. Anxiety by argonaut on Wednesday December 30, 2015 @09:18AM (#10812115)
Re:The A.I. Anxiety by sawwit on Friday January 01, 2016 @09:36AM (#10821968)
I never said am certain, I'd assign both cases about the same probability. Since I've already stated why I think the malevolent simple AI scenario such as a paperclip maximizer is plausible, you haven't stated why you find it implausible. That is why I asked.

Also, what does the Intermediate value theorem have to do with it?

Re:The A.I. Anxiety by argonaut on Sunday January 03, 2016 @02:59AM (#10829903)
You asked why I was certain. So I asked why you were certain. The onus is on the people putting out apocalyptic theories to justify the apocalyptic scenario (extraordinary claims require extraordinary evidence - a central tenet of the scientific method).

We are going to have a normal-intelligent AI before a superintelligent AI.