a) we’ve done that, which is cool. Now let’s figure out how to control it.
b) you can’t get your gmail for a bit while we reboot the DC. That’s probably okay.
b) except that is not how these things go in the real world. What actually happens is that initially it’s just a risk of the agent going rogue, the CEO weighs the multi-billion dollar cost vs. some small-seeming probability of disaster and decides to keep the company running until the threat is extremely clear, which in many scenarios is too late.
(For a recent example, consider the point in the spread of Covid where a lockdown could have prevented the disease from spreading; likely somewhere around tens to hundreds of cases, well before the true risk was quantified, and therefore drastic action was not justified to those that could have pressed the metaphorical red button).
The blast radius of such decisions are large enough that this option is not trivial as you suggest.