Most of the people I know who believe that the AI labs should be shut down because AI is too dangerous are for "general technological advancement" and for our becoming more capable as a species. We make an exception for AI because of how capable it might soon become and because no one has a plan for how to control an AI that has become "better at reality" than people are. If someone were to come up with such a plan -- an actually good plan -- almost all of us would withdraw our objection to the continuation of frontier AI research.

If someone were to come up with a good plan for how to control an AI such that people stay in control even as the AI becomes more capable than any person or group of persons, I would not worry much that Altman or Amodei would ignore the plan: both of them know that current plan that they have is severely lacking and that adopting an actually-good plan would reduce the danger that they themselves personally (and their families) incur from the operation of the corporation that they run.

A nice analogy is the serial killer: a large fraction of serial killers are relieved when they are finally caught. They know that what they are doing is wrong, they just can't stop themselves because (for them) killing feels really really good. Analogously, I think Altman and Amodei would be relieved if the government shut down OpenAI and Anthropic because they know that what they are doing puts everyone including themselves and their families at fatal peril, they just can't stop because (for the kind of person who becomes head of an AI lab in the first place) it feels really really good to be the leader of a trillion-dollar-valuation company that is transforming society -- and all the personal power and attention they get from that role.