Whether this particular scenario is plausible or not, the overarching concept behind it is path dependence. Path dependence implies that humankind may only have one chance to go through certain transformations. For instance, the Internet, begun in 1966, took nearly 30 years to become the behemoth force it is now. No single corporation could have pulled it off because of the timeline alone.

Consider however that we've been doing this transformation thing since we began as a species.

https://en.m.wikipedia.org/wiki/Path_dependence

I'm glad you're thinking about the long-term prospects of humanity. Your point is well-made that a technological regression, at this point, would be exceedingly difficult for future humans to reverse.

If humans do not spread life beyond earth, then life will not survive long-term. It does feel like we must succeed in this first great technological expansion, or we might never succeed, and fall prey to Fermi's Demon.

I sort of left two additional open questions:

- How long can the current era last? I.e. are we default-alive or default-dead?

- Am I not thinking laterally enough about what pre-industrial civilizations could have done?

Consider that disasters scale fractally. You're much more likely to see a failed AI superintelligence destroy the one supplier of critical infrastructure product XYZ, than directly destroying the world. Its fractal self similar on a smaller scale of hubris, bad specification, bad planning, bad back out plans, poor estimation, bad competitive risk taking.

The AI super intelligence that destroys our only global manufacturer of XYZ is very likely to be written in Excel in a spreadsheet convincing the execs that selling junk bonds and opening an ambitious new plant is a great idea, fast forward five years and humanity literally has no CPUs, or no flash memory, or no spark plug insulators. Its very unlikely to be black and white mad 1950s movie mad scientist writes bad LISP code that makes a humanoid-ish robot self aware making it crack open the smallpox vials intentionally. Its also worth considering that "we" are pretty good at destruction with existing dumb tools, it seems unlikely we'd need intelligent tools when simpler cheaper dumb tools cause equal chaos. It didn't take an AI to destroy HP, or Bell Labs, for example.

Its statistically possible that the first AI super intelligence failure will instantly vaporize the planet in the same sense that its thermodynamically possible for all molecules in a box of air to randomly be found on one side of the box, however unlikely. In a similar way its very likely we'll have AI destroying individuals, companies, industries, markets, countries, long before it destroys the species.

I think you haven't done even cursory research into A.I super-intelligence before dismissing it.

Try reading Nick Bostrom's "Superintelligence: Paths, Dangers, Strategies"[1] and then I think you will change your mind.

[1]http://www.amazon.com/Superintelligence-Dangers-Strategies-N...

I don't dismiss it, I just believe it to be unlikely. I am in the middle of Bostrom's book right now.