News for hackers. Stuff that matters.
Re:The Myth of a Superhuman AI by DubiousPusher on Wednesday April 26, 2017 @08:35PM (#14208256)
I think a compelling argument is the variance in intelligence among humans themselves. A cabal of AIs 2x as intelligent as your average "genius" could be quite formidable were they to decide to act against the favor of humankind.
Re:The Myth of a Superhuman AI by adamlett on Thursday April 27, 2017 @01:31AM (#14209389)
Yes, this is why the nerds rule over the jocks in high school. Oh, wait…
Re:The Myth of a Superhuman AI by Qwertious on Thursday April 27, 2017 @03:21AM (#14209802)
That's different, because there are specific rules against arms races in schools, that people typically adhere to. If there was a battle scheduled in advance, I'd bet on the nerds.

But what's more, if they both have guns then everything's about even. Strength doesn't really help if you've been shot, and most high-schoolers don't have much experience shooting guns.

Re:The Myth of a Superhuman AI by ghaff on Thursday April 27, 2017 @09:20AM (#14211216)
>most high-schoolers don't have much experience shooting guns

Actually a lot do but probably not so much in the places that readers/commenters here live.

Re:The Myth of a Superhuman AI by TeMPOraL on Thursday April 27, 2017 @05:02AM (#14210148)
They don't win in high school. But later in life, jocks find themselves employed by nerds.
Re:The Myth of a Superhuman AI by Retra on Wednesday April 26, 2017 @09:14PM (#14208432)
But anyone half that smart would know not to give an unproven system absolute, unchecked control over their critical infrastructure, which would basically be a requirement to causing problems humans can't easily overcome.
Re:The Myth of a Superhuman AI by manicdee on Wednesday April 26, 2017 @09:57PM (#14208616)
The Machines will come to control our infrastructure through a series of small decisions that seemed like a good idea at the time.

Computer control of farming decisions such as watering and fertilising. Link that in with insolation and wind prediction to minimise waste or maximise productivity.

This happens in the cloud thanks to insolation prediction being tied into wide-area networks of solar farms.

More such networks evolve independently, then one day some bright spark realises that prediction and control in domain X can be combined with control and prediction in domain Y to produce greater efficiencies.

Linking reproductive health and food production to maximise water savings, for example.

Next thing you know the computers decide that human population is best maintained in the order of hundreds of millions rather than billions.

And then we find out that Terminator was actually propaganda from the human rebels who didn't like the computers telling them not to breed, and the nuclear war was Skynet protecting its desired humans from the rebels, not protecting itself.

Re:The Myth of a Superhuman AI by nradov on Thursday April 27, 2017 @12:44AM (#14209226)
The limiting factor in farm productivity is not decision making by human farmers. Anyone who has been farming for years already makes pretty close to optimal decisions based on the available data about crop prices, weather, soil status, labor rates, etc. The ones who make bad decisions end up bankrupt, and leave farming or work as hired farmhands. And there are so few decision-making farmers left anyway that automating the decision making part of their jobs won't achieve any meaningful cost savings.
Re:The Myth of a Superhuman AI by alexwebb2 on Thursday April 27, 2017 @10:25AM (#14211657)
> Anyone who has been farming for years already makes pretty close to optimal decisions based on the available data about crop prices, weather, soil status, labor rates, etc.

I'm not sure if you're including software analysis in your notion of "based on the available data", but if not, then this would absolutely not be true - unaided humans, even veteran growers who've been managing the same fields for decades, do not make anywhere close to optimal decisions. They make good ones, generally, and can be long-term profitable with relative ease, but it's absolutely not even close to what you get when you bring intelligent software into the mix.

Re:The Myth of a Superhuman AI by nradov on Thursday April 27, 2017 @11:48AM (#14212371)
Sure of course good farmers are taking advantage of the best available software analysis tools. The point is that those tools already work well enough that even if superhuman AI actually becomes available at some point (and that might never happen) it won't necessarily make any better decisions than what human farmers are already doing today.
Re:The Myth of a Superhuman AI by kpil on Thursday April 27, 2017 @01:30AM (#14209384)
Maybe Amazon will own 90% of the farms because the have the cheapest robots and best algorithms and statistics.
Re:The Myth of a Superhuman AI by nradov on Thursday April 27, 2017 @01:51AM (#14209458)
John Deere and their competitors are far, far ahead of Amazon.com in farm automation. There's a huge difference between warehouse picking robots and farm equipment. Being cheaper doesn't matter, it's ROI that counts.
Re:The Myth of a Superhuman AI by Danihan on Thursday April 27, 2017 @11:25AM (#14212181)
Except Amazon could easily acquire John Deere.
Re:The Myth of a Superhuman AI by ShannonAlther on Thursday April 27, 2017 @01:02PM (#14213032)
Just briefly scanning the numbers on Wikipedia, Amazon.com could acquire John Deere, but it'd be highly unlikely unless they expected that a change in management would yield implausibly high returns.
Re:The Myth of a Superhuman AI by Danihan on Thursday April 27, 2017 @01:34PM (#14213280)
I suppose my point is, they could acquire the knowledge needed to do cutting edge farm automation if that's something they wanted.

And it should be something they want.

Re:The Myth of a Superhuman AI by nradov on Friday April 28, 2017 @10:16AM (#14219536)
I'm sure Jeff Bezos is eagerly awaiting your advice on Amazon.com acquisition targets. Farm automation is a great strategic fit with web services and online retailing, right?
Re:The Myth of a Superhuman AI by Danihan on Friday April 28, 2017 @11:53AM (#14220436)
It's a rather straightforward step in controlling the supply chain when you are pushing something like Amazon Fresh.

I suppose they also aren't a hardware manufacturer? Or a logistics company?

Re:The Myth of a Superhuman AI by MarkPNeyer on Wednesday April 26, 2017 @10:13PM (#14208693)
Just how much faith do you have in your politicians?

By using existing political structures, and a mix of rhetoric, they have managed to get a lot more power than a typical person.

Re:The Myth of a Superhuman AI by DubiousPusher on Thursday April 27, 2017 @11:21AM (#14212149)
Nice point. The US presidential election has had me thinking about how a smart AI might decide instead to manipulate humans into it's will with misinformation campaigns. That of course would require an AI of human comparable intelligence.
Re:The Myth of a Superhuman AI by ben_w on Thursday April 27, 2017 @12:39PM (#14212853)
I believe it doesn't necessarily need to outsmart the /average/ human. Consider this scenario:

* Someone makes an AI which scores only 85 on IQ tests, but which can run in real time on common consumer hardware.

* It gets installed on every phone, PC, and tablet.

* Someone, possibly the AI itself if is the type to generate its own goals, sets all instances of the AI the goal of convincing their user of some belief or policy or whatever.

* Everyone with IQ<85 is convinced, nobody with an IQ>85 is convinced.

* 16% of the population believe whatever it is the goal-setting mind wanted them to believe, at marginal cost. An election is swung, a bank suffers a run, a pyramid scheme makes millions, whatever it is.

Re:The Myth of a Superhuman AI by MarkPNeyer on Friday April 28, 2017 @04:06PM (#14222639)
the ability to emit words on the screen gives a human-like intelligence all kinds of abilities

especially if the human-like intelligence doesn't need to pay rent, fear the police, follow laws, sleep, eat ... etc

Re:The Myth of a Superhuman AI by ben_w on Tuesday May 02, 2017 @07:22PM (#14251609)
In the absence of counter examples, it's reasonable to assume a truly human-like intelligence needs as much down-time and sleep as a real human, and that it can be as effectively coerced with the threat of simulated prison if it breaks the rules.

But the point about food and rent is very economically important if they ever get to be cheaper than human labour.

Re:The Myth of a Superhuman AI by crusso on Wednesday April 26, 2017 @10:00PM (#14208639)
Would you imagine that a computer smarter than the smartest human and having the understanding of computer software that probably only a computer would have couldn't conceal what it's doing as it creates trojans and viruses that can give it control over every piece of connected equipment everywhere?
Re:The Myth of a Superhuman AI by Retra on Thursday April 27, 2017 @12:08AM (#14209109)
All of that is moot when you can just turn off the power.
Re:The Myth of a Superhuman AI by pjc50 on Thursday April 27, 2017 @06:25AM (#14210444)
Can you?

The AI, when it becomes popular, is going to be a business-critical system. Potentially even intertwined with life-critical systems like self-driving cars and self-flying planes. If it isn't, then in "hidden malevolent hacker" phase it can just install trojans to make those systems dependent on it.

Petrol production is hazardous to our future. Can you "just turn it off"? Yes; shut down a fairly small number refineries and the system will run dry within a week. CO2 emissions problem solved! It's just that, you know, people were depending on being able to get food delivered by truck.

Obviously the first malevolent AI is going to hide itself behind an offshore opaque-ownership company and hire a good set of PR and lobbying firms. Can you turn off Skynet when it's reported as "critical to national security" in all the news sources?

Re:The Myth of a Superhuman AI by Retra on Thursday April 27, 2017 @07:56PM (#14215859)
So you're supposing that we have honed and selected an AI so wonderfully adept at valuing human needs that we make most human life dependent on it, yet it will spontaneously come to disregard the value of human life? And despite the fact that we would naturally require it to be sufficiently transparent about it's valuations in order to actually ensure we get the results we want, it will also develop this animosity in secret?

That seems a bit contradictory to me. And if not, highly contrived and unlikely.

Re:The Myth of a Superhuman AI by crusso on Thursday April 27, 2017 @10:40PM (#14216527)
"And despite the fact that we would naturally require it to be sufficiently transparent about it's valuations"

That's not necessarily true at all. Let's say we come up with a way to model neurons in silicon like a very complex neural net to the point of emergent consciousness. We might not be able to look into memory location 0xFFA79348 and get the "human value" int to see if it is high enough.

It could very well be that we don't really understand the weighting in the net or the implications of those weights because it's too complicated. Even current neural nets of non-trivial complexity can be impenetrable to evaluate outside of the net itself.

Re:The Myth of a Superhuman AI by Retra on Friday April 28, 2017 @06:50PM (#14223797)
Transparency doesn't require that you be able to read its memory directly, just that it is able to sufficiently justify the decisions that it makes.
Re:The Myth of a Superhuman AI by crusso on Saturday April 29, 2017 @02:19PM (#14227882)
Like we can justify all decisions made by super genius people who are extraordinarily good at concealing their activities when their cognitive processes and abilities on the Internet and with other forms of communication would be beyond our comprehension?

I think that you overestimate our ability to understand what we can create. Even Deep Neural Net researchers consider the neural nets to be a black box that they don't really understand. Further... unlike human beings who are locked into their genetics and their neuronal pathways - an AI will have the instant ability to improve itself. It will be able to rewrite its own software faster than we can comprehend. In the hours from the time the AI would be created until a researcher even might realize what he had created, the AI will effectively have had years or tens of years of contemplation time and self-enhancement time. If the researcher goes to sleep with plans to work with the new AI the next morning, the AI will effectively have hundreds of years of time to plan and improve itself. If it has any kind of Internet access, it will be able to take over systems around the world, possibly parallelizing its own cognitive abilities and possibly backing up its consciousness in case the researcher decided to eventually wipe it.

The genie will be out of the bottle so fast that human beings will almost assuredly be caught flat-footed.

I always shake my head when I see sci-fi shows where the humans (Stargate, Star Trek, etc.) come up against advanced AI opponents. It's only through control of the script and plot contrivances that the humans have a chance. People just won't be able to compare to thinking and self-modifying AI.

https://www.ted.com/talks/sam_harris_can_we_build_ai_without...

Re:The Myth of a Superhuman AI by pjc50 on Friday April 28, 2017 @10:42AM (#14219735)
> AI so wonderfully adept at valuing human needs that we make most human life dependent on it, yet it will spontaneously come to disregard the value of human life?

This is an argument often deployed against "corporations" or "capitalism", both of which aren't quite intelligent but are definitely systems that we're dependent on and capable of disregarding inconvenient lives.

The AI doesn't have to hate humans, it just has to find that they get in the way of its quarterly targets.

Re:The Myth of a Superhuman AI by Klockan on Thursday April 27, 2017 @04:06AM (#14209956)
That wont help, we know how to turn off humans from across the world with the click of a button yet we still haven't managed to create peace on earth.
Re:The Myth of a Superhuman AI by lgas on Thursday April 27, 2017 @03:50AM (#14209893)
If it was successfully concealing its actions from you, what reason would you have to turn off the power?
Re:The Myth of a Superhuman AI by Retra on Thursday April 27, 2017 @07:51PM (#14215829)
If it were successfully concealing its actions, what wrong would it be doing? If all the bad things someone does has no effect on anyone else, then who cares if they do them?
Re:The Myth of a Superhuman AI by crusso on Thursday April 27, 2017 @10:42PM (#14216535)
Because it could be concealing its actions until it controls enough infrastructure or military power to launch an attack or prevent people from being able to shut it down.

What we don't know can hurt us.

Re:The Myth of a Superhuman AI by Retra on Friday April 28, 2017 @06:55PM (#14223825)
It is not 'concealing its actions' if it is taking control of infrastructure and military power.

Just like I'm not 'concealing my actions' by putting a blindfold over your eyes. My action is "putting a blindfold over your eyes", which must be very apparent for the entire action to have any value. Now if I hide that action by putting on an invisible blindfold, then you've got nothing to complain about anyway.

Re:The Myth of a Superhuman AI by crusso on Saturday April 29, 2017 @08:50PM (#14229621)
Why would it be so obvious if it understands enough about strategy to use subterfuge?

When the US government unleashed Stuxnet, did it announce what it was going to do?

Re:The Myth of a Superhuman AI by TeMPOraL on Thursday April 27, 2017 @05:01AM (#14210143)
Batteries exist. And so do small-scale power generators like PV cells, windmills and, well, diesel generators.
Re:The Myth of a Superhuman AI by ben_w on Thursday April 27, 2017 @06:12AM (#14210395)
Counter-examples exist every time any critical infrastructure is hacked: if the systems were truly proven, they wouldn't be hackable.

Proof is hard, good enough gets deployed.

Re:The Myth of a Superhuman AI by szc on Thursday April 27, 2017 @12:48AM (#14209244)
This makes me want to find and watch "Colossus: The Forbin Project" (1970s SCI-FI)
Re:The Myth of a Superhuman AI by epx on Wednesday April 26, 2017 @09:10PM (#14208413)
Yes, but it is noteworthy that most geniuses are shortsighted in other respects. They probably would not survive in prehistory.
Re:The Myth of a Superhuman AI by pacaro on Wednesday April 26, 2017 @10:43PM (#14208806)
It's not clear to me how true this is. There are definitely individuals that exhibit savant skills in one area and mediocre to poor skills in other areas. But there are also true polymaths who can turn their hands to just about anything, I suspect that those people would do well on aggregate in any period of human history
Re:The Myth of a Superhuman AI by obstinate on Wednesday April 26, 2017 @11:27PM (#14208973)
Depends on the style of genius. Napoleon, Genghis Kahn, Alexander the Great . . . they seem to have done pretty well for themselves.
Re:The Myth of a Superhuman AI by wand3r on Wednesday April 26, 2017 @10:24PM (#14208740)
Obviously some did.
Re:The Myth of a Superhuman AI by jacquesm on Wednesday April 26, 2017 @11:03PM (#14208870)
At least long enough to reproduce.