I would think Pascal's Wager can be applied to that. If Musk is wrong, being extra careful with AI will just mean slightly slower development. If he's right and we don't take it seriously, humanity is wiped out. Why take the chance?

Pascal's Wager is considered a joke for good reason. It's not any less a terrible argument here than in philosophy.

If you go outside, you might get struck by a car. Why take the chance?

Awareness of a risk isn't very helpful if it doesn't come with an understanding of the odds and the costs and benefits involved.

Pascal's Wager is something you either make or don't make based on your own understanding of the odds of each outcome and the costs of each outcome.

Clearly you have a different understanding than Elon.

I thought that pascal's Wager only works with infinite reward/ punishment.

End of humanity is in the grand scheme of things something trivial. It's just happens and then nobody suffers. so there must be a probability low enough to balance it out.

AI won’t have to do much— humanity already on course to destroy itself.