News for hackers. Stuff that matters.
Re:The Scariest, Most Boring Armageddon by VLM on Sunday January 31, 2016 @02:02PM (#11007496)
Consider that disasters scale fractally. You're much more likely to see a failed AI superintelligence destroy the one supplier of critical infrastructure product XYZ, than directly destroying the world. Its fractal self similar on a smaller scale of hubris, bad specification, bad planning, bad back out plans, poor estimation, bad competitive risk taking.

The AI super intelligence that destroys our only global manufacturer of XYZ is very likely to be written in Excel in a spreadsheet convincing the execs that selling junk bonds and opening an ambitious new plant is a great idea, fast forward five years and humanity literally has no CPUs, or no flash memory, or no spark plug insulators. Its very unlikely to be black and white mad 1950s movie mad scientist writes bad LISP code that makes a humanoid-ish robot self aware making it crack open the smallpox vials intentionally. Its also worth considering that "we" are pretty good at destruction with existing dumb tools, it seems unlikely we'd need intelligent tools when simpler cheaper dumb tools cause equal chaos. It didn't take an AI to destroy HP, or Bell Labs, for example.

Its statistically possible that the first AI super intelligence failure will instantly vaporize the planet in the same sense that its thermodynamically possible for all molecules in a box of air to randomly be found on one side of the box, however unlikely. In a similar way its very likely we'll have AI destroying individuals, companies, industries, markets, countries, long before it destroys the species.

There are no comments on this comment yet.