News for hackers. Stuff that matters.
Re:The Dawn of AI: How to ensure the promise outweighs the p... by return0 on Thursday May 07, 2015 @11:34AM (#9505797)
> Fifth, as advanced AI, it could develop actual feelings, including sadism, greed, etc.

I get that some AIs will have simulated reward signals, but letting them go haywire would just be bad design. We have nuclear heads, but we do not design them to explode at random. Now, our own reward mechanisms arise from the grander plan to keep organic life alive. Theirs have no underlying motive, and i don't think it goes the other way around, i.e you can't create a grander plan starting from the rewards. One would need a mathematical proof for the latter.

Re:The Dawn of AI: How to ensure the promise outweighs the p... by DanAndersen on Thursday May 07, 2015 @11:43AM (#9505860)
I think there's some over-anthropomorphization of the AI here, in terms of it picking up human qualities as if they were Things outside our brains that we possess.

But I feel like this particular argument is (please correct me if I'm wrong) saying something like "Well of course we wouldn't let them go crazy! That'd be ridiculous!" I agree -- a sensible solution would be to have good design and introduce safeguards. That's the point of people like Bostrom when they're advocating for this, because before they started writing about this, it seems like a lot of people weren't even thinking it could be a problem. How many systems will be built without good design, because they were made to solve a short-term problem. Will the finance bots doing transactions at the microsecond scale in order to maximize the assets of some particular company have been built to not let them go haywire?

Re:The Dawn of AI: How to ensure the promise outweighs the p... by coldtea on Thursday May 07, 2015 @12:03PM (#9506043)
>I think there's some over-anthropomorphization of the AI here, in terms of it picking up human qualities as if they were Things outside our brains that we possess.

I don't see any issue with anthropomorphization with regard to intelligent being (at any level).

To have an issue with that would be to assume that humans are more special than other life forms (and AI would be like a life form in this sense).

Animals can have empathy, anger, anxiety and other traits we associate with humans (and why wouldn't they? It's not like our brains are unique, they're just "more"), so AI could too.

Re:The Dawn of AI: How to ensure the promise outweighs the p... by coldtea on Thursday May 07, 2015 @12:00PM (#9506010)
>I get that some AIs will have simulated reward signals, but letting them go haywire would just be bad design.

The problem with this is that for real AI, you don't have control over what they feel. It's not like some program of the sixties or seventies AI, were stuff was "simulated", it can be modelling a real neural network that works like the brain, and into which you don't have much control over what will develop.

That is, the kind of AI people are afraid of is not "simulated", it's actually thinking for itself.