>History

Right, I learned from Napoleon about the dangers of AI...

The large land mammals were doing pretty well too, until a bipedal superintelligence showed up.

History is a get thing to learn from and a very poor thing to assume is going to repeat.

You would have more weight behind your words if there was any sort of precedent for AI as it exists today being super intelligent, or that you could make a case for some method by which you could suddenly ramp out of control without our notice.

But as is, all you're doing is making theories, and there's not much of any precedent for any of them.

> that you could make a case for some method by which you could suddenly ramp out of control without our notice.

lots of things ramp up without our notice. Like CO2 in the atmosphere, or Ozone depleting chemicals, or PFAS in the water, microplastics in the ocean. I don't think it's a small list.

CO2 has not ramped up without notice, nor did ozone nor pfas. There are active efforts too handle each of them, and the only reason we aren't cracking down on most of those harder is that the utility they provide to human beings is far greater than the damages right now.

Sure they did. They are now active efforts to handle them, but after they became issues, not while they were ramping up.

You've just provided evidence that humans are the most dangerous, not anything else on this planet.

I mean, so far yes. At the same time we have tons of people spending billions of dollars to make sure that's not the case, and I'm pretty sure they are going to succeed.

That's tautological and irrelevant. Yes, humans are most dangerous to every other life form on the planet. That's why humans rule the Earth, and not some other life form. But in terms we care about - humans being a danger to humans - we have some handle on it.

The worry with superintelligent AI is that it'll dethrone us, and automatically make it the most dangerous thing on the planet. In particular, more dangerous to us than we already are to each other.

If you want to make a claim that we won't even see change coming, maybe the large mammals->humanoids transition isn't the greatest example. That took several hundreds of thousands of years.

They did, to some extent, see it coming (in observing a threat becoming more prevalent and dangerous). But they did not have a way to stop humans. Meanwhile, I'm just staring at the power switch on my computer, wondering how we'd stop AI.

(This is not an argument for unrestrained AI - we should safeguard, absolutely. It is a counterpoint to your "we won't see it coming" argument)

We see climate change coming (and here) and yet seeing it coming isn't doing much to stop it.

This is we talk about Moloch commonly on HN. We see systems that are going to have obvious bad outcomes for most individuals, and be unaligned with what most humans want, and yet the system persists and even grows worse. This is how I believe it will be with AI too. It will make the rich insanely rich and powerful even at the risk of destroying us all.