Getting uranium, the hardware to have nukes, and all the research and production around it is hard without being detected. Starting to code AI on a Mac and a few good GPUs or even a cluster without being detected is crazy easy. Even if they start to treat the most powerful GPUs as small nukes and only a few can have them it’s super easy to steal, transport, store, use… Nukes not so much.

For frontier AI models you don't need a few GPUs or a small cluster. You need thousands of them, you need them all connected with high bandwidth and you need a lot of power. It's not something you can do in your garage or undetected.

There is some risk that over time people will be able to do more with less, and eventually you can train an AGI with a few dozen A100s. If that happens I agree there's nothing you can do, but until then there is a chance.

What is the difference between a weather model, a nuclear bomb simulation, and training a LLM?

At the end of the day all these are just calculations, of which you need thousands of processors to do with high accuracy. Pretty much what you're suggesting is a global police force to ensure we're performing the 'correct calculations'. Having to form an authoritarian world computer police isn't much of a solution to the problem either. This just happens to work well in the nuclear world because it's hard to shield and the average person doesn't have a need for uranium.

If we had to have global policing of every large computing cluster and everything that runs there, that does not feel to me like a huge overreach or something that could easily slide into totalitarianism.

It would be better if we didn't need to do this, but I don't see a less intrusive way.

Of course, if you don't think there is a real existential threat then it's pointless and bad to do this, it all depends on that premise.

Intelligence is a real potential existential threat, just ask every large mammal as of about 10k years ago, save those in Africa.

The question is, is an authoritarian nightmare that much better than getting tossed into the meat grinder?

> The question is, is an authoritarian nightmare that much better than getting tossed into the meat grinder?

If the choice was between those two extremes, I'd take the authoritarian nightmare.

Extinction is forever, dictatorships end.

> Ukraine and other former soviet republics voluntarily gave them up after the breakup of the USSR.

And no country will ever make the same mistake again. Because the west promised to protect them from aggression. But when the time came to do just that - everyone involved dusted off their copy of Budapest memorandum and handed it over to the lawyers.

Sorry for digression. Carry on.

For sure. If the Russian federation breaks up into a bunch of independent states we will have a huge problem.

It was not very fair from civil war already, with the aborted coup. Could you imagine if the Wagner group became a nuclear power??

So... 44% of countries that have obtained nuclear weapons did so after the Nuclear Weapons Treaty. China alone added 60 warheads in 2022. It doesn't matter that it works 90% of the time - if a single entity is able to break out of the prohibition, it's game over.