News for hackers. Stuff that matters.
Re:OpenAI safety leader quits, warning AI company's culture ... by nradov on Sunday October 04, 2026 @12:22AM (#49950603)
Who is "we"? The GPUs and training algorithms get more efficient all the time. In a few years, creating effective LLMs isn't going to require massive GPU clusters.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:03AM (#49950804)
We is society through government via regulation. I don’t think GPUs or training will get that efficient that fast absent a distillation target provided by the easily accessible frontier lab APIs.

Further, even if you are right, so what. Is that a reason to just accept bad public policy? That’s like saying, anyone can learn how to make smallpox at home with a basic lab set up so we should just ignore any safety measures.

Re:OpenAI safety leader quits, warning AI company's culture ... by eddythompson80 on Sunday October 04, 2026 @01:40AM (#49951001)
> Is that a reason to just accept bad public policy?

Not necessarily, but it should probably inform that public policy. I think the problem is no one knows what the public policy should be assuming that scenario is true. Even if you, somehow, regulate away massive GPU cluster training making such future training impossible, existing models are already here. Further already training smaller models for things like images, speech, and other specialties is cheaper than the bigger models.

I agree that we need some regulations like everything else, but it’s not clear to me what the right policy should be. I think the European ai act is a fine start, but it’s clearly not enough nor does it necessarily limits the training portion just the application portion. Not to mention that the requirements there can be summarized into something like “you have to be careful, and show evidence you tried to be careful”.

Re:OpenAI safety leader quits, warning AI company's culture ... by lukan on Sunday October 04, 2026 @03:58AM (#49951668)
“you have to be careful, and show evidence you tried to be careful”.

That sounds reasonable. If applied to OpenAI and their agents multiple times breaking out of bad secured sandboxes, it should be enough.

But limiting the training?

There really is china and they have a different approach I suppose. But it is possible to talk with them.

Re:OpenAI safety leader quits, warning AI company's culture ... by eddythompson80 on Sunday October 04, 2026 @11:14AM (#49954606)
> That sounds reasonable. If applied to OpenAI and their agents multiple times breaking out of bad secured sandboxes, it should be enough.

Does it? To me it seems reasonable for OpenAI to argue they did try to be careful evident by the sandbox, they just made a mistake. Almost every 0day is categorized by something like that. We haven’t had a long history of establishing a negligence charge to security bugs. Could you be sued because you didn’t demonstrate “carefulness” and used Linux which is not written in a memory safe language and has had multiple CVEs before? How complicated should the chain of an exploit be to demonstrate “carefulness” to the courts?

> training

OP was the one suggesting that training could be controlled because massive gpu clusters could be regulated the way a nuclear power plant could. If you assume training costs won’t drop, then it’s feasible I guess. However, unlike a nuclear reactor, the final training result isn’t a radio active material, but rather an ordinary file that anyone can load and use for inference.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:08PM (#49955726)
It seems really clear at this point China has only been able to keep pace by distillation farms.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:07PM (#49955720)
My entire point is that we are currently on a path to a duopoly which isn’t just expected to cover training and serving models but the entire knowledge economy. That could be mitigated by limiting their train and inference capacity to a specific percentage of total available compute. That would ensure other operators could compete in the market. Instead we are letting OpenAI literally contract to buy all available ram to the point that Apple can’t buy ram and had to cut their hardware configurations.
Re:OpenAI safety leader quits, warning AI company's culture ... by nradov on Sunday October 04, 2026 @02:55PM (#49956723)
Apple could have bid higher but chose not to. Most modern desktop software is extremely memory inefficient. Developers have largely ignored this in recent years because RAM was so cheap. But there's tremendous opportunity for improvement with a little optimization work.

We used to run Microsoft Word and other popular applications with 8 MB RAM and it worked fine.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @04:57PM (#49957770)
No. Once the contract is signed it will be breach of contract. It’s not always a bidding process.

I am working on reducing ram requirements to run models but the weights are already compressed to the edge of the Shannon entropy boundary and might resist further compression.

Re:OpenAI safety leader quits, warning AI company's culture ... by nradov on Sunday October 04, 2026 @01:38AM (#49950990)
What a silly comparison. LLMs have nothing to do with smallpox.

Computing always gets cheaper and faster over time. We can argue about the exact rate of improvement but the results are inevitable and uncontrollable.

Re:OpenAI safety leader quits, warning AI company's culture ... by lukan on Sunday October 04, 2026 @03:47AM (#49951610)
Short reminder from the guidelines:

'When disagreeing, reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3." '

Also LLM's have something to do with smallpox as a unrestricted LLM will happily guide any wannabe terrorist in how to make them.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:11PM (#49955755)
But the amount of available compute has been contractual bought by the frontier labs such that you can’t get equivalent compute even if you had the money. That is market lock up and is a policy choice. Free markets require market access. Anticompetitive contracting destroys markets and innovation. We don’t allow that in any other commodity market and we shouldn’t allow it for GPUs. You are not allowed to legally corner the market for silver or soybean futures.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:20PM (#49955831)
I am reasoning from analogy. Smallpox is dangerous so we have regulations that limit who can do research and how they do it. If LLMs are similarly dangerous we could do the same. As the government did with mythos.
Re:OpenAI safety leader quits, warning AI company's culture ... by nradov on Sunday October 04, 2026 @02:57PM (#49956741)
Nah. There is no actual evidence of such danger. Fears about Mythos were completely overblown.