My most likely doom scenario is that a frontier model convinces the CIC to press the nuclear retaliation button. Maybe this will only kill 90% of humans, but as far as I am concerned, that’s good enough.

not that good

If the Commander In Chief is that gullible we are already doomed. Under that logic we should stop having babies in case selection results in a "superpersuader" capable of convincing the president to launch the nukes unbidden.

I think a realistic scenario is mass murders triggered by AI psychosis. Some weirdos start an AI cult and end up killing a bunch of randos because the AI said doing so would break the matrix or something similar. I am actually surprised this hasn't happened yet.

Great job, now you just told AI how to get away with it.

Well, Ziz did happen, although I think you mean the other AI psychosis

I'm not so sure that the argument that some humans will survive would placate anyone who is worried about a catastrophic outcome from AI development. You could make the same argument for climate collapse or global nuclear war, but I also don't think that's gonna make anyone feel better about those scenarios

i thought virus engineering was a scenario already confirmed

Yeah, but the counter is:

"Bro just don't synthesize the virus that wipes out humanity if the ai tries to make one"

Because humans are famously excellent at never running files that someone posing as a trusted colleague with a solution to an urgent problem gives them. /s

The phishing attempts will be wild.

"Hey there the boss wants you to do the gain of function on this one, here is a [deepfaked] copy of his face and voice telling you so

I see a lot of people from Silicon Valley who take that threat seriously, but not a whole lot of biologists who do.

I can't get anyone outside of SV to take AI threats seriously. Even the HF attack seems to glaze the eyes. It's all just too sci-fi feeling. I just end up flapping my little arms and squawking.

Besides, biologists are usually acutely attuned to what they correctly perceive as the bigger threat (climate change,) which they also can do much more about, professionally speaking, than glitching over AI.

Worrying about AI is our job. Suppressive fire so they can go back to their dying reefs and starving polar bears. Next year's tragedy.

A big bulldozer wont even slow down while driving right through a bit barn, and that is a realistic scenario where humans wipe out all AI, quickly if desired, realy a lot of big bulldozers, cheap too $200~ $400hr.

You're conveniently ignoring the coordination problem. You'd have to take them all out more or less contemporaneously, and if climate change is anything to go by, good luck getting coordinated global action to avert a profitable risk.

I wasn't ignoring the probl, but hey the solution is simple, just get AI to build a bulldozer scheduling app and gather up all the contact info for the bulldozer people, hire a multilingual call center and have the problem take care of itself.

True, itll take a lot of human support from its fascist incubators.