News for hackers. Stuff that matters.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Saturday October 03, 2026 @11:09PM (#49950248)
There is lots of knowledge that requires a license to operate commercially. We could just do this.
Re:OpenAI safety leader quits, warning AI company's culture ... by 01100011 on Saturday October 03, 2026 @11:51PM (#49950442)
So you cede the frontier of human progress to illicit organizations and other nations?
Re:OpenAI safety leader quits, warning AI company's culture ... by daveguy on Sunday October 04, 2026 @12:25AM (#49950625)
These LLMs are not the frontier. They are a tool. A tool that doesn't need to be able to write a sonnet to be useful.
Re:OpenAI safety leader quits, warning AI company's culture ... by harshitaneja on Sunday October 04, 2026 @01:12AM (#49950843)
We don't know that. We don't know if this particular kind of tool can do "useful" things without also developing the ability to write a sonnet. And they are absolutely a frontier. I am not suggesting we should continue building them just because they are, there are many technologies which could have been built had we thrown the amount of resources we have here and there is a case to be made for not doing it at the pace we are or if at all. But we can do that without diminishing what exists.
Re:OpenAI safety leader quits, warning AI company's culture ... by daveguy on Sunday October 04, 2026 @02:36PM (#49956570)
I just mean the business case is much more likely to come from treating it like a tool than a proto-diety. Frontier of business, not frontier of impressive demo. Also, would argue they aren't frontier of research, in that a research frontier expands in more directions than just the latest hot thing in silicon valley. See also, LeCun, Chollet, etc. "These labs aren't the entirety of the frontier" would have been more accurate. And any of those frontier directions could be the next hot thing.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:32AM (#49950955)
So an illicit organization is going to operate a frontier scale data center and do $1b training pulling power from the grid without detection?
Re:OpenAI safety leader quits, warning AI company's culture ... by 01100011 on Sunday October 04, 2026 @11:15AM (#49954610)
They very likely will via shell companies in jurisdictions outside the west. Are you suggesting we go to war to stop them? The massive datacenters are needed to serve models to millions. Criminal orgs don't need massive data centers anyway.

As Bruce Schneier recently discussed, law and tax law are code, just like source code. LLMs are great at finding holes in them. Illicit organizations looking to launder funds are most certainly interested in what AI can do for them.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @12:56PM (#49955607)
It’s not a binary choice. There are more options than let the labs run wild and capture the full stack and then the whole knowledge economy or make them entirely illegal and move all activity to the black market. We can regulate to limit their ability to operate to just providing utility inference: make them divest codex, Claude code and any apps; they can only be an API that others build on. Limit their ability to buy compute to a specific amount of available supply so other companies are able to buy compute. Limit their ability to accumulate private training data and require that they make their training data publicly available for others to use after a certain period of time. Make them legally obligated to publish their weights so others can. All of this would increase competition and ensure that they don’t establish dominance over society.

Then we could require comprehensive logging of every tool call, model trace, chain of reasoning, and even tensor propagation all of which would be spot inspected like the CFTC does with commodity trading and settlement. We could have embedded auditors with specific risk analysis metrics like large banks do. We could limit tool calls to dedicated sandbox’s with a blanket prohibition on AI accessing user space. We could create a parallel internet for agents so they are only able to access Secure Enclave. Even if these measures aren’t 100% perfect they would reduce the risk.

Re:OpenAI safety leader quits, warning AI company's culture ... by 01100011 on Sunday October 04, 2026 @11:24PM (#49960352)
Reducing which risk? The risk that the models of big American companies cause harm? What if they abide by some rules while models of others remain unfettered? That also carries risk. I appreciate the wish to have tame AI which only does good things but I'm afraid we're already beyond that.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Monday October 05, 2026 @11:08AM (#49965862)
I would frame it as inverse. Rather than aiming at reducing the risk of harm, our Ami should be matching the incentive to get personally rich with a corresponding risk of personal bankruptcy or criminal liability. That wouldn’t just incentivize safety, it would incentivize abstaining from activities with insufficient confidence of avoiding even long tail risk.

More importantly it would incentivize the entire industry at the same time by injecting caution into the yolo speed run we have now. No frontier lab or single user hobbyist would take imprudent risk in the hope that they get rich if they also risk the full weight of consequences.

Re:OpenAI safety leader quits, warning AI company's culture ... by RandomLensman on Sunday October 04, 2026 @03:12AM (#49951434)
No, why would that be the consequences of regulation?