If people had started to worry about the long-term effects of carbon output before it was already widespread, the Industrial Revolution would have been strangled at birth (the road from wood fires to solar panels leads through coal-burning steam engines; refuse to ever burn coal in large quantities and the chain is broken), and we would have stumbled around banging the rocks together until evolution optimized general intelligence out of existence or the sun autoclaved the biosphere.

Nothing in the world is more dangerous than premature attempts at safety.

Who said anything about refusing to burn coal in large quantities? The benefits of the industrial revolution may well have outweighed the costs, and no one is saying it was a bad thing – or likewise, that developments in AI are a bad thing.

But are you saying there's absolutely nothing that could've been done better in the industrial revolution? With some foresight we might have thought to develop solar power more urgently, for example, but no one thought we needed to. Worrying just a little might have changed that.

The point is not to stop progress, just to approach with caution and be mindful of what the long-term implications are.

I'm saying there is nothing that would in fact have been done better by premature worry about global warming. If your argument is that any sequence of actions will be less than optimal - compared to, say, the actions that could have been carried out by a hypothetical omniscient entity of infinite wisdom and benevolence - then that is certainly true but irrelevant. But the previous argument was that convincing actual humans in the nineteenth century to start worrying about global warming would have been net beneficial, and I'm pointing out it would have been disastrous.

And that, mind you, is still with the anachronistic application of 21st-century knowledge to the matter. If people in the 19th century had actually tried to figure out what they should be worrying about if they were going to worry two centuries prematurely, they would likely have come up with something completely different.

>With some foresight we might have thought to develop solar power more urgently, for example, but no one thought we needed to.

With some foresight, people were trying to develop solar, nuclear, and fusion all the way back in the '60s and '70s, but political interests ensured that research funding was reallocated towards shorter-term political projects.

You aren't getting the comparison.

It's akin to someone back then worrying about the long-term effects of steam/water vapor.

You don't know if vapor is actually a problem, or will become one, and you don't have any data in any field to back up your concerns.

Which would have been perfectly reasonable.

When pharma companies produce a new drug which appears to have a positive effect, they don't know that it will cause problems and don't have any data to back up any concerns.

So do they rush full-steam-ahead into the unknown? Do you unleash the drug to whomever wants it, without regulation? No, of course not; you approach with caution, you hold clinical trials, you collect data as you go and you look for potential problems before the drug is widespread.

Arguably a similarly controlled approach would have helped in the case of carbon output and will help in the case of AI. It's not about fear mongering or preventing progress, it's about discussing ways to approach with caution and solve problems (and minimise the damage they cause), if and when they arise.

Arrhenius predicted global warming in 1896: http://en.wikipedia.org/wiki/Svante_Arrhenius#Greenhouse_eff...

Ah, Fantastic! He led the creation of the field of climatology if I recall.

If someone can do forecasting of AGI with similar empiricism that would be revolutionary and welcomed.

Arrhenius probably did no empirical research in this field. He devised a generalized formula based on theoretical considerations and indirect measurements (of the moon's appearance). It would take ~60 years before a quorum of researchers believed his formula could provide accurate real-life predictions.

Moravec's 70s papers are roughly comparable to Arrhenius's formula. The basic computational power predictions (a minor extension of Moore's law) still seem to be basically correct. There's a reasonable debate to be had about whether more interesting predictions made by Moravec/Kurzweil/etc. have been validated or not.

Solving problems that don't exists is rational? Human progress would be severely diminished by such actions.