Re:The Unreasonable Reputation of Neural Networks
by Houshalter on Sunday January 17, 2016 @09:21PM
(#10922186)
Yes that's one of the important topics covered in the book. The AI probably would probably value resources and self preservation.
Resources help it fulfil whatever it's goal is, and the more the better. That includes energy and computing power. Which it could gain a lot of by converting the mass of the solar system into a dyson swarm. Or covering the Earth with solar panels or fusion reactors and comuters. Etc.
Self preservation would be optimized by destroying anything that has even a tiny chance of being a threat to it. Even if humans are harmless, we could build other AIs like it which might compete with it. It's best to just destroy us.
The problem is the AI doesn't need to want to eliminate us. It will likely be entirely indifferent to us. We just need to be in it's way.
Resources help it fulfil whatever it's goal is, and the more the better. That includes energy and computing power. Which it could gain a lot of by converting the mass of the solar system into a dyson swarm. Or covering the Earth with solar panels or fusion reactors and comuters. Etc.
Self preservation would be optimized by destroying anything that has even a tiny chance of being a threat to it. Even if humans are harmless, we could build other AIs like it which might compete with it. It's best to just destroy us.
The problem is the AI doesn't need to want to eliminate us. It will likely be entirely indifferent to us. We just need to be in it's way.