>Fantasizing about alternative realities where all the leaders of the world unite and halt technology development without any of them secretly defecting and developing it further isn’t very useful to discuss. These things aren’t going to happen.

it has kinda worked for nuclear weapons

In the nuclear case multiple powers raced to develop it as fast as possible and as much as possible.

That’s the inverse of what I said.

"We" (Israel, probably with US assistance) assassinate "rogue" (outside of existing nuclear powers) nuclear scientists (e.g. from Iran). So to take this analogy to its conclusion you have to accept that part of the pact would be assassinating people for doing computer science research or building computer systems that will be used for AI.

Nuclear weapons come with the immense benefit that purifying uranium is so challenging, expensive, and difficult to hide that we don't even bother to protect the information on how to build a nuke. They also have a second benefit of being impossible to test without the whole world knowing you did.

AI has no such hurdles.

No, the AI compute supply chain is actually quite similar to the nuclear supply chain. Building chips capable of training or large-scale inference is extraordinarily complex, expensive, has a large footprint, and has a few chokepoints (rare earths mining, HBM manufacturing, and ultraviolet lithography, where there is literally one company in the world capable of building the relevant outputs) where we in fact already have governance mechanisms for things like export controls.

The "world" isn't just what humans currently make of it. You deny the possibility of alternatives for society based on what amounts to ignorance.

Transitioning from the current "order of things" to one compatible with the existence of things like AI is a necessity, not a "fantasy".

When you imagine people as savages to be outdone in their savagery, you become one yourself. Spend more effort on finding ways not to.

> The "world" isn't just what humans currently make of it. You deny the possibility of alternatives for society based on what amounts to ignorance.

Ignorance of what? All of the previous successful attempts where the entire world reorganized itself into a single world government?

> Transitioning from the current "order of things" to one compatible with the existence of things like AI is a necessity, not a "fantasy".

I think you’re putting the cart before the horse. You’re assuming it’s necessary and then making another leap to assume that being necessary means it must also be possible.

You're acting like international treaties (or cartels) don't exist. Yes, these are structures where parties have incentives to defect, and we counter those by creating incentives not to, with checks and sanctions.

Additionally, you don't need all the leaders of the world to agree to anything, you only need US and China.

Additionally x2, if you agree with the common stance that China makes the vast majority of their progress by distilling US models, you don't even need them to agree to anything.

A ban on AI looks nothing like a ban on almost every other technology that has come before it.

You're really suggesting the following bans. A ban on computer science and algorithms. A ban on any device that can play games that aren't encrypted or DRM locked from an approval authority. A ban on any kind of unregulated computing without government approval and licensing.

>China makes the vast majority of their progress by distilling US models

Eh, this is a budgetary decision, not a technological one. Why spend billions when the US can spend billions and you can spend millions. China more than has the power to do what we're doing if they chose to do so, their biggest limitation is speed of making new compute, not money or brains.

> You're really suggesting the following bans. A ban on computer science and algorithms. A ban on any device that can play games that aren't encrypted or DRM locked from an approval authority. A ban on any kind of unregulated computing without government approval and licensing.

You know what, fuck it, that's exactly what I'm suggesting. Let's ban personal computers. The only consumer hardware allowed should be a HTTPS client directly burned on the chip that can only connect to government servers. Absolute all timer of a straw man. This is probably the last comment I'm writing on this site, this place has become unsufferable.

Believe it or not, there already are things you're not allowed to do with computers that you otherwise own, and we have ways of enforcing them other than trusted computing -- the threat of putting you in a cell for a few decades usually does the job. That's without getting into the fact that you're not going to train 1T-sized models on your laptop, and billion dollar GPU farms aren't precisely hard to keep track of.