Movies and strawmen alike.

Most people think continued existence of humanity is good, which includes environmentalists. But there are some people who either think (a) avoiding suffering is overwhelmingly important, to the extent that a universe without life is ideal or (b) the world would be better off without humans because of the vast destruction we cause. I'm not claiming this is a common view, but it doesn't have to be a common view to result in destruction if destruction gets sufficiently straightforward.

For some writing in this direction, see https://longtermrisk.org/risks-of-astronomical-future-suffer... which argues that suffering-focused people should not try to kill everyone primarily because they are unlikely to succeed and https://philarchive.org/archive/KNUTWD ("Third, the negative utilitarian can argue that losing what currently exists on Earth would not be much of a loss, because of the following very plausible observation: overall, sentient beings on Earth fare terribly badly. The situation is not terrible for every single individual, but it is terrible when all individuals are considered. We can divide most sentient beings on Earth into the three categories of humans, non-human animals in captivity, and wild non-human animals, and deal with them in turn. ...")

This doesn't really answer my question. These are arguments for why threat actors might hold the belief that "humanity is cancer", but it doesn't actually provide evidence that this is a credible threat in the real world.

The two people I cited at the end are examples of philosophers who think a lifeless world is preferable to the status quo and to futures we are likely to get. These aren't arguments for why someone might hold the belief, but evidence that some people do seriously hold the belief.

(Another example here would be https://en.m.wikipedia.org/wiki/Aum_Shinrikyo)

Fair enough. I don't doubt there are people in the world that hold this view. I just don't know how credible they are as a threat to humanity.

Elon Musk and Joe Rogan were just complaining about one of them on Rogan’s podcast a day or two ago. He was on the cover of Time magazine, but I can’t find it on Time’s site. If you dig up that podcast on YT you should be able to see it.

The calculation of considering something is a credible threat directly corresponds with the ease of implementation.

If I said "I am going to drop a nuke on jebarker" that's not a credible threat unless I happen to be a nation state.

Now, if I said "I'm going to come to jebarkers house and blast him with a 12 gauge" and I'm from the US, that is a credible threat in most cases due to the ease of which I can get a weapon here.

And this comes down to the point of having a magic machine in which you could ask for the end of the world. The more powerful that machine gets the bigger risk it is for everyone. You ever see those people that snap and start killing everyone around them? Would you want them to have a nuke in that mood? Or a 'lets turn humanity in to grey goo' option?

The problem with this analysis is that we have complete uncertainty about the probability that AGI/ASI will be developed in any particular time frame. Anyone that says otherwise is lying or deluded. So the risk equation is impossible to calculate for AGI/ASI wiping out humanity. And since you appear to be evaluating the damage as essentially infinite, i.e. existential risk, you're advocating that as long as there's a greater than zero probability of someone using that capability then the risk is infinite. Which is not useful for deciding a course of action.

No, no need to calculate infinities. Instead lets imagine something with an IQ within one exponent higher of ours. This is exceptionally difficult to even imagine as below 70 we consider a person disabled in some form or another. Once we get in the range of IQ over 170 or so it really starts to break. When we get into multimodal self learning systems there may be vast amounts of knowledge encoded into its networks that we simply don't know the right question to ask as humans. Attempt to think of what other AIs at or near that level could ask each other.

Our different systems of intelligence keep falling back to ideas of human intelligence and its limitations. Things like we don't multitask well, our best attention is given to one thing at a time. Things like our highest bandwidth senses are our ears and eyes, and even then they are considerably bandwidth constricted. Human limits like our abilities cannot be horizontally scaled in an easily measurable way, and adding more people massively increases the networking costs of accomplishing a goal.

If for some reason the smartest AI can only reach the levels of the smartest human beings (which at least to me makes no sense), then this is still massively powerful, as of this point pesky human problems like 8 hours of sleep are not needed. If human level AI power can be shrunk down to near cellphone size and power consumption, which again doesn't seem out of the realm of physics, lays down the framework for an intelligence explosion at the rate which we can print chips out and assemble them.

But the problem still stands that we can't estimate how likely any of this is to occur. So in the overall risk equation we have a term that is essentially completely uncertain and therefore varying that term can yield any arbitrary risk level you want.