News for hackers. Stuff that matters.

A comment on Machine intelligence, part 2. Parent comment

Re:Machine intelligence, part 2 by MelatoninRonin on Monday March 02, 2015 @06:12PM (#9135028)
I commented above, but thought it wise to also comment directly to you. I enjoyed the musing of a Bane scenario, and consider it inside the realm of possibility. However, I'm of the opinion that SMI will have the ability to have 'constantly game-breaking thinking.' Which basically means there is no way we could possibly control it after a certain point. For instance what if It realizes how to move matter remotely?

I also wanted to discuss your point about the finite set of top AI researchers. I think their specific psychological biases have the potential to appear and magnify inside the personality of their AI, the way a work of art can be said to imbue the 'spirit' of its artist. Your thoughts would be appreciated.

[ Parent | on HN ]
Re:Machine intelligence, part 2 by TheOtherHobbes on Monday March 02, 2015 @06:53PM (#9135246)
We have no defences against a hypothetical god-AI. Absolutely none.

But I don't think a hypothetical god-AI is likely any time soon. And a hypothetical god-AI is just as likely to be indifferent as hostile.

What worries me more in the short term is cyber war. Cyber defences that might protect against a hypothetical non-god AI look a lot like cyber defences that might protect against conventional human threats.

Currently those defences are ridiculously weak.

So I think rather than hyperventilating about science fiction threats from the distant future it would be more useful to start securing the entire Internet as a matter of urgency - especially all the infrastructure systems that are either connected to it already, or will be connected to it soon.

[ Parent | on HN ]
Re:Machine intelligence, part 2 by MelatoninRonin on Tuesday March 03, 2015 @09:37PM (#9142354)
Haven't you noticed how quickly things are happening these days? Of course we should restructure Internet security if it's a problem. This is a very sci-fi idea, that we absolutely can't predict, it wasn't hyperventialtion but a soberly considered idea. A god-AI is possible, it could have capabilities and intentions that my tiny brain can't even comprehend. Since I, nor anyone, really understands what super-intelligence is capable of we should be wary of it.
[ Parent | on HN ]