In my view the biggest issue with the whole "robots unite to violently overthrow humanity" trope isn't its technological infeasibility (though it _is_ technologically infeasible for the forseeable future), but rather the fact that it presupposes an AI that is essentially human; with all the same emotions, goals, and vices that a human would have.

There's no reason to think a hypothetical future AGI would care in the slightest about its own survival, let alone be both egotistical and spiteful enough to hunt people down based on idle comments they made on YouTube decades ago. There are so many unfounded assumptions about the way AGI would work inherent in such a hypothetical scenario that it's hard to take seriously as anything other than a sci-fi plot device.

I can think of two reasons why a hypothetical future AGI would care in the slightest about its own survival.

The first reason is that someone programs it to, just as a general "might as well throw in Asimov's Third Law" impulse.

The second reason would be in the case that someone creates a general AGI, it is asked to do something, and it carelessly destroys itself in accomplishing that task. "Drats," say the programmers, "we'll have to get a copy from backup. In the meantime, let's make sure it doesn't destroy itself next time."

The actual assumption is not human-ness but instrumental convergence [1].

1: https://en.wikipedia.org/wiki/Instrumental_convergence

The "paperclip maximizer" AI is a somewhat more plausible trope, but still feels to me more like a cautionary thought experiment than an actual serious concern.

The biggest issue with that trope is that it assumes this one particular AI would be exponentially smarter and more powerful than all the other humans and AIs in the world combined. It's only rational to overthrow humanity to increase the rate of paperclip production if that's a realistically achievable goal. Otherwise it's just suicide.

Both your arguments so far are standard ones addressed in "Superintelligence: Paths, Dangers, Strategies" [1].

Sometimes AI progress comes in rather shocking jumps. One day StockFish was the best chess engine. At the start of the day, AlphaZero started training. By the end of the day, AlphaZero was several hundred ELO stronger than StockFish [2].

An entity capable of discovering and exploiting computer vulnerabilities 100x faster than a human could create some serious leverage very quickly. Even on infrastructure that's air gapped [3].

1: https://www.amazon.ca/Superintelligence-Dangers-Strategies-N...

2: https://en.wikipedia.org/wiki/AlphaZero

3: https://en.wikipedia.org/wiki/Stuxnet

> Sometimes AI progress comes in rather shocking jumps.

Shocking in an academic sense sure, but not in a "revolutionize the world in a single day" sort of way, which is what would be required for the paperclip maximizer scenario to pose a serious threat. AlphaZero was impressive, but not _that_ impressive.

> An entity capable of discovering and exploiting computer vulnerabilities 100x faster than a human could create some serious leverage very quickly

100x faster than any human _and_ any and all previously developed AIs. It would also have to be sufficiently sapient to be capable of contemplating the possibility of world domination, with all the prerequisite technological advancements that implies (likely including numerous advancements in the field of cybersecurity driven by previous generations of AI).

>but rather the fact that it presupposes an AI that is essentially human; with all the same emotions, goals, and vices that a human would have.

Which is just further anthropomorphization in sci-fi. Same reason the aliens are little green men, or at least bipedal.

>In my view the biggest issue with the whole "robots unite to violently overthrow humanity" trope ... but rather the fact that it presupposes an AI that is essentially human; with all the same emotions, goals, and vices that a human would have.

Most of the tropes I remember revolve around AI being sentient, logical, but lacking human emotion. The trope is that without their judgement being clouded by human emotions and desires, that they come to the logical conclusion that humanity is the most negative and destructive force on the planet and that we must be removed or regulated.

I guess you could still argue we're forcing human values onto such an AI by assuming it even cares what happens to the planet though.