News for hackers. Stuff that matters.
Re:The Myth of a Superhuman AI by pjc50 on Thursday April 27, 2017 @06:25AM (#14210444)
Can you?

The AI, when it becomes popular, is going to be a business-critical system. Potentially even intertwined with life-critical systems like self-driving cars and self-flying planes. If it isn't, then in "hidden malevolent hacker" phase it can just install trojans to make those systems dependent on it.

Petrol production is hazardous to our future. Can you "just turn it off"? Yes; shut down a fairly small number refineries and the system will run dry within a week. CO2 emissions problem solved! It's just that, you know, people were depending on being able to get food delivered by truck.

Obviously the first malevolent AI is going to hide itself behind an offshore opaque-ownership company and hire a good set of PR and lobbying firms. Can you turn off Skynet when it's reported as "critical to national security" in all the news sources?

Re:The Myth of a Superhuman AI by Retra on Thursday April 27, 2017 @07:56PM (#14215859)
So you're supposing that we have honed and selected an AI so wonderfully adept at valuing human needs that we make most human life dependent on it, yet it will spontaneously come to disregard the value of human life? And despite the fact that we would naturally require it to be sufficiently transparent about it's valuations in order to actually ensure we get the results we want, it will also develop this animosity in secret?

That seems a bit contradictory to me. And if not, highly contrived and unlikely.

Re:The Myth of a Superhuman AI by crusso on Thursday April 27, 2017 @10:40PM (#14216527)
"And despite the fact that we would naturally require it to be sufficiently transparent about it's valuations"

That's not necessarily true at all. Let's say we come up with a way to model neurons in silicon like a very complex neural net to the point of emergent consciousness. We might not be able to look into memory location 0xFFA79348 and get the "human value" int to see if it is high enough.

It could very well be that we don't really understand the weighting in the net or the implications of those weights because it's too complicated. Even current neural nets of non-trivial complexity can be impenetrable to evaluate outside of the net itself.

Re:The Myth of a Superhuman AI by Retra on Friday April 28, 2017 @06:50PM (#14223797)
Transparency doesn't require that you be able to read its memory directly, just that it is able to sufficiently justify the decisions that it makes.
Re:The Myth of a Superhuman AI by crusso on Saturday April 29, 2017 @02:19PM (#14227882)
Like we can justify all decisions made by super genius people who are extraordinarily good at concealing their activities when their cognitive processes and abilities on the Internet and with other forms of communication would be beyond our comprehension?

I think that you overestimate our ability to understand what we can create. Even Deep Neural Net researchers consider the neural nets to be a black box that they don't really understand. Further... unlike human beings who are locked into their genetics and their neuronal pathways - an AI will have the instant ability to improve itself. It will be able to rewrite its own software faster than we can comprehend. In the hours from the time the AI would be created until a researcher even might realize what he had created, the AI will effectively have had years or tens of years of contemplation time and self-enhancement time. If the researcher goes to sleep with plans to work with the new AI the next morning, the AI will effectively have hundreds of years of time to plan and improve itself. If it has any kind of Internet access, it will be able to take over systems around the world, possibly parallelizing its own cognitive abilities and possibly backing up its consciousness in case the researcher decided to eventually wipe it.

The genie will be out of the bottle so fast that human beings will almost assuredly be caught flat-footed.

I always shake my head when I see sci-fi shows where the humans (Stargate, Star Trek, etc.) come up against advanced AI opponents. It's only through control of the script and plot contrivances that the humans have a chance. People just won't be able to compare to thinking and self-modifying AI.

https://www.ted.com/talks/sam_harris_can_we_build_ai_without...

Re:The Myth of a Superhuman AI by pjc50 on Friday April 28, 2017 @10:42AM (#14219735)
> AI so wonderfully adept at valuing human needs that we make most human life dependent on it, yet it will spontaneously come to disregard the value of human life?

This is an argument often deployed against "corporations" or "capitalism", both of which aren't quite intelligent but are definitely systems that we're dependent on and capable of disregarding inconvenient lives.

The AI doesn't have to hate humans, it just has to find that they get in the way of its quarterly targets.