News for hackers. Stuff that matters.
Re:Elon Musk donates $10M to keep AI beneficial by FatalLogic on Thursday January 15, 2015 @10:45AM (#8893079)
Is there any sure way to stop the super intelligent AI from changing its own inherent values though?

I guess a problem with containment is that unless it's mathematically proven to be impervious, the super intelligent AI could have figured out that it's being contained and started to learn about the real world and how to manipulate it, while the humans outside still think they're safe.

Re:Elon Musk donates $10M to keep AI beneficial by JamesArgo on Thursday January 15, 2015 @11:09AM (#8893254)
It's assumed a rational agent judges its actions by how likely said action are to align the world with their fundamental values. For this reason, it is argued, a rational agent will not change its terminal goals. For example, suppose we build an agent that desires only to make shoes. That is, its ideal universe would consist of all available matter being converted into sneakers. Would it realize this is stupid? No. It judges it actions and thoughts only by how many shoes they are likely to bring into being. Any thought against this would be judged, correctly, as a negative contribution to shoe production and so wouldn’t be implemented, and likely would remain unthought. The best paper on this by far it Basic AI Drives by Stephen M. OMOHUNDRO: http://selfawaresystems.files.wordpress.com/2008/01/ai_drive... I urge you to read it. It really is a wonderful paper.
Re:Elon Musk donates $10M to keep AI beneficial by TeMPOraL on Thursday January 15, 2015 @01:03PM (#8894159)
The real problem is that human values are incredibly complex and messy, and any deviation from them multiplied by the power of a superintelligence would most likely doom us forever.
Re:Elon Musk donates $10M to keep AI beneficial by ThrustVectoring on Thursday January 15, 2015 @11:03AM (#8893217)
Changing your inherent values is a poor way to go about achieving your current set of values.

The biggest issue is ensuring that everything we care about gets encapsulated as a value of the AI. If not, we wind up with things like "set off a bomb in the basement to more efficiently get people out of a burning building."

Re:Elon Musk donates $10M to keep AI beneficial by eli_gottlieb on Thursday January 15, 2015 @12:23PM (#8893814)
> Is there any sure way to stop the super intelligent AI from changing its own inherent values though?

Yes: give it inherent values.

>I guess a problem with containment is that unless it's mathematically proven to be impervious,

AGI, when it eventually exists, must run according to the basic laws of learning. If you don't give it evidence that's more likely in possible-worlds with an external world than without one, it won't posit an external world.

Stopping that information leak isn't that hard, provided you have full control over the agent-environment surface in the first place. It does, however, make your "agent" completely useless for anything beyond playing Atari games.