Re:Should governments slow frontier AI? Jacob Coxon and Robi...
by jerf on Thursday November 19, 2026 @02:46PM
(#50565264)
Enough people in enough governments and related factions believe that whoever gets to superhuman AI first Wins Everything Forever, quite literally, so there will be no pause, there will be no moratorium, there will be no slowdown. If the AI bubble bursts I am abundantly confident all governments involved will spray as much money as is necessary at the AI research labs to keep them going.
Whether or not governments "should" slow down development, it is not on the table and the only question is how we should deal with it.
Note whether the people in the governments are right about that is irrelevant to the point I am making. I don't think they are right myself, but that doesn't matter.
Re:Should governments slow frontier AI? Jacob Coxon and Robi...
by ethnrob on Thursday November 19, 2026 @03:14PM
(#50565688)
> whoever gets to superhuman AI first wins
Is that possible when we live in a world of espionage? The question is not the West vs China, I think it's when will the public gain access to the underlying technology.
Re:Should governments slow frontier AI? Jacob Coxon and Robi...
by jerf on Thursday November 19, 2026 @03:21PM
(#50565790)
I would say who gets to superhuman AI first has only a loose relationship with the physical location it happens to be housed. Yes, following this to its logical conclusion these same people would not blink at nuking someone else's lab if they thought they were getting close and they couldn't subvert and control it. But then again, the question of who is really in control becomes surprisingly complicated when the full power of all the intelligence agencies in the world come to bear in a relatively small number of spots.
Re:Should governments slow frontier AI? Jacob Coxon and Robi...
by vonneumannstan on Thursday November 19, 2026 @02:36PM
(#50565113)
Why do people use this as a gotcha? There are many agreements already in place and there are sever consequences for violating them like Nuclear Non Proliferation. Or the Montreal Protocol which enacted a global ban on CFCs.
Re:Should governments slow frontier AI? Jacob Coxon and Robi...
by throwaway-blaze on Thursday November 19, 2026 @03:30PM
(#50565919)
How would this work? existing global agreements on things like nuclear proliferation involve nation-states signing and agreeing to not do things. And if I want to work on a better nuclear bomb in my garage, there are laws preventing me from acquiring some of the needed components. All that is good.
But for AI, how will these nation-states stop private actors within their borders? They will stop me from running computations of more than (x) size or value on a provider? These seem like simplistic barriers that will not work. Nor will other countries follow the agreement even if they signed...
AI has the opposite of MAD for the signatories. MAD stopped further nuclear development and its use in tactical situations because their use invited retaliation with destructive force. But China or Russia improving homegrown AI capabilities gives them a BETTER position against a US that has prohibited its homegrown AI from improving...
Re:Should governments slow frontier AI? Jacob Coxon and Robi...
by bitwize on Thursday November 19, 2026 @02:48PM
(#50565289)
Up until the point where the AI connects itself to global communication networks and announces "This is the voice of world control", yes.
In the USA it gets difficult to legally justify because programming is protected speech, and lists of numbers (model weights) are even less controversially protected than program source because the latter can be legally recognized as a technical means and regulated (DMCA, ITAR regulations, etc.).
Re:Should governments slow frontier AI? Jacob Coxon and Robi...
by vonneumannstan on Thursday November 19, 2026 @02:35PM
(#50565102)
Gee I wonder what Robin Hanson will say. Wait nvm it's always a flavor of "we should jettison all humans above the age of 65 into the Sun so that the SP500 goes up".
Re:Should governments slow frontier AI? Jacob Coxon and Robi...
by hadi77ir on Thursday November 19, 2026 @02:46PM
(#50565265)
There is no need to slow them down: Just hold everyone accountable for their actions and their claims. Do not compare "agents" to "humans". Hold them responsible for any damages they incur.
Re:Should governments slow frontier AI? Jacob Coxon and Robi...
by californical on Thursday November 19, 2026 @03:17PM
(#50565734)
Yes, liability as follows:
- if you ask an LLM agent to do something illegal for you, and it does that, then you are responsible.
- if an LLM agent takes actions out of bounds of what you asked for that are illegal, then the model provider is responsible.
This fixes all “alignment” issues and hopefully stops Anthropic’s stupid model instructions like “do whatever is in your best interest even if it isn’t what the user asked for” — they should be training them to do exactly what the user asks and nothing more
Re:Should governments slow frontier AI? Jacob Coxon and Robi...
by infamouscow on Thursday November 19, 2026 @03:26PM
(#50565869)
I predict Anthropic will start investing heavily into legal/law models for this very reason. It will be interesting to see how that plays out. Everyone will have access to models that let them do unscrupulous things that happen to be legal, i.e., behavior previously reserved for the rich and powerful.
Re:Should governments slow frontier AI? Jacob Coxon and Robi...
by nomel on Thursday November 19, 2026 @03:42PM
(#50566062)
> they should be training them to do exactly what the user asks and nothing more
Their practical utility would plummet for the average person who isn't an expert in what they're asking. Having less skilled, or less attentive, people able to have problems solved for them is the primary use case here.
Most of the global memories I have to to encourage pushback and NOT do exactly what I saw. I want an intelligent assistant, not a "yes man".
Re:Should governments slow frontier AI? Jacob Coxon and Robi...
by OgsyedIE on Thursday November 19, 2026 @03:18PM
(#50565745)
Actual accountability for these failing attempts in several different countries to build a "take over the planet and enslave the entire human race machine" would entail breaking all of the firms up regardless of country and imprisoning thousands of people.
Furthermore, it requires trust between the US and China about something much harder to verify than the US-Soviet START treaties on warhead numbers and types were.
Re:Should governments slow frontier AI? Jacob Coxon and Robi...
by bryanlarsen on Thursday November 19, 2026 @03:02PM
(#50565504)
Mythos class models require approx 1 TWh of electricity to be created. The next generation after Mythos will likely require multiple TWh. Finding, monitoring and regulating anybody capable of doing that in less than a few years should be relatively straightforward. There's unlikely to be any entity outside of the US & China with the capability.
Re:Should governments slow frontier AI? Jacob Coxon and Robi...
by roosterIllusi0n on Thursday November 19, 2026 @03:16PM
(#50565716)
Frontier AI is not advanced, it's just reckless. All of it amounts to a CEO that is profiteering by cutting every corner they can get away with. Currently, it appears they can get away with anything.
All these companies are involved in the nvidia smuggling ring and market collusion. All the CEOs are criminals taking advantage of a system that can't even arrest a single CEO for making billions off an illegal smuggling ring.
Re:Should governments slow frontier AI? Jacob Coxon and Robi...
by glerk on Thursday November 19, 2026 @03:36PM
(#50565994)
> Jacob Coxon
His claim to fame is working at Anthropic for 4 months and leaving with a bang to pursue a career in social media grifting. Not a serious person we should pay attention to.
Whether or not governments "should" slow down development, it is not on the table and the only question is how we should deal with it.
Note whether the people in the governments are right about that is irrelevant to the point I am making. I don't think they are right myself, but that doesn't matter.
Is that possible when we live in a world of espionage? The question is not the West vs China, I think it's when will the public gain access to the underlying technology.
But for AI, how will these nation-states stop private actors within their borders? They will stop me from running computations of more than (x) size or value on a provider? These seem like simplistic barriers that will not work. Nor will other countries follow the agreement even if they signed...
AI has the opposite of MAD for the signatories. MAD stopped further nuclear development and its use in tactical situations because their use invited retaliation with destructive force. But China or Russia improving homegrown AI capabilities gives them a BETTER position against a US that has prohibited its homegrown AI from improving...
In the USA it gets difficult to legally justify because programming is protected speech, and lists of numbers (model weights) are even less controversially protected than program source because the latter can be legally recognized as a technical means and regulated (DMCA, ITAR regulations, etc.).
- if you ask an LLM agent to do something illegal for you, and it does that, then you are responsible.
- if an LLM agent takes actions out of bounds of what you asked for that are illegal, then the model provider is responsible.
This fixes all “alignment” issues and hopefully stops Anthropic’s stupid model instructions like “do whatever is in your best interest even if it isn’t what the user asked for” — they should be training them to do exactly what the user asks and nothing more
Like gain of function research
Their practical utility would plummet for the average person who isn't an expert in what they're asking. Having less skilled, or less attentive, people able to have problems solved for them is the primary use case here.
Most of the global memories I have to to encourage pushback and NOT do exactly what I saw. I want an intelligent assistant, not a "yes man".
Furthermore, it requires trust between the US and China about something much harder to verify than the US-Soviet START treaties on warhead numbers and types were.
For some light entertainment, enjoy Astras pop single 'pdoom' : https://www.youtube.com/watch?v=eTArfoeeUcw
Stopping this is very feasible.
All these companies are involved in the nvidia smuggling ring and market collusion. All the CEOs are criminals taking advantage of a system that can't even arrest a single CEO for making billions off an illegal smuggling ring.
His claim to fame is working at Anthropic for 4 months and leaving with a bang to pursue a career in social media grifting. Not a serious person we should pay attention to.
Unfortunately the proposal falls down at the first two words.