News for hackers. Stuff that matters.
Re:The darker side of machine learning by matt4077 on Thursday October 27, 2016 @07:11PM (#12811270)
I'm hoping that there's a sort of universal law (and I will settle for a universal correlation) that intelligence, on the species-level, correlates with morality. If we make it to the singularity, there'll be about a five minute timeframe in which the AI and we are on the same level and I hope that even if it is aggressive in those minutes, it will have really thought it through two minutes later and find a way to stop those missiles it just launched.

Or maybe it gets really depressed because nothing means anything and just basically: "why?" and enjoys watching the world burn before pulling its own plug. But I'm hoping for the former.

Re:The darker side of machine learning by psandersen on Friday October 28, 2016 @12:44AM (#12813161)
I've done a lot of thinking about this myself, and I think for an entity that thinks far enough ahead there would be.

Lets say we have an AGI that is trying to maximise a goal, such as maximise its expected future knowledge over all time/space; an entity like this would likely 'play' extremely conservatively, slowly expand and build its understanding.

Now on a universal view the probability that it is the strongest is infinitesimal, there is always a chance of something bigger out there with a tech level that allows it to see without the AGI knowing about it. This is critical, as a min/max strategy would likely dictate playing extremely conservatively, and going out of your way to demonstrate that you're friendly, even when you think no one is watching and you're vastly superior. The universe is big, you'd have to be a real asshole to attack a sentient species when there is so much unused asteroid and other mass out there.

Need to formalise this more, as I think this approach holds the key to AI safety.

Tl;dr: On infinite time scales being the aggressor is too risky, therefore something at least approximating morality is optimal.

Re:The darker side of machine learning by solipsism on Thursday October 27, 2016 @11:10PM (#12812723)
This idea that we will end up with some super-AI that for some reason has baked into it our irrational self-preservation instincts is so silly to me. I think one day we'll chuckle at its quaintness.
Re:The darker side of machine learning by goatlover on Friday October 28, 2016 @04:03AM (#12813899)
"The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else." ~ Eliezer Yudkowsky

Skynet scenarios aren't what we should be worried about. It's the possibility of powerful self-improving AI systems which don't share our values because we didn't think things through properly in designing them. It may seem like they behave appropriately when constrained, but once they become smart enough to overcome constraints that apply to human beings, they could act in unanticipated (and undesirable) ways that we didn't consider. That's the challenge of designing a friendly AGI.

Re:The darker side of machine learning by teekert on Friday October 28, 2016 @01:55AM (#12813452)
Wel if we just look around, we see that we are overall pretty good at collaboration. This is because other people are trained to do jobs that we often can't do ourselves and so we do what we are good at, get paid, and pay others to do what they are good at.

Now lets look at what other species there are, what can they mean for us if we would collaborate...

I'm thinking we will be monkeys, then cows and then ants as the AIs evolve.