News for hackers. Stuff that matters.
Re:OpenAI safety leader quits, warning AI company's culture ... by pcthrowaway on Saturday October 03, 2026 @10:16PM (#49949964)
Trolley problem:

- If you allow to trolley to proceed, there's a 50% chance it will run over every human on the planet

- But if you flip the switch, it takes the long way around, possibly bankrupting the trolley company. And you have a legal obligation to the shareholders to prevent that from happening at all costs.

Re:OpenAI safety leader quits, warning AI company's culture ... by MichaelDickens on Saturday October 03, 2026 @11:12PM (#49950263)
That's why it's important that the trolley company was established as a non-profit. And if it does take funding, investors' returns will be capped at 100x.

(...wait)

Re:OpenAI safety leader quits, warning AI company's culture ... by simoncion on Sunday October 04, 2026 @04:11AM (#49951749)
It's also why even the trolley company's CEO agrees that it's important that -should the board determine that the CEO is fantastically untrustworthy and unethical- the board is able to fire said CEO and replace him with someone that's trustworthy and ethical.

(...wait.)

Re:OpenAI safety leader quits, warning AI company's culture ... by NexRebular on Sunday October 04, 2026 @11:58AM (#49955037)
> ...someone that's trustworthy and ethical.

So... a language model?

Re:OpenAI safety leader quits, warning AI company's culture ... by austhrow743 on Saturday October 03, 2026 @10:46PM (#49950119)
If you flip the switch then the trolley still proceeds and there’s still a 50% chance every human on the planet gets run over. You’re just not the one at the wheel.
Re:OpenAI safety leader quits, warning AI company's culture ... by 01100011 on Saturday October 03, 2026 @10:48PM (#49950138)
Humanity has discovered a way to create a form of intelligence using math. This knowledge is not going back in the box.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Saturday October 03, 2026 @11:08PM (#49950238)
But that math cant run without massive GPU clusters. We don't have to allow openai or anthropic access to those anymore than we have to allow a company to operate nuclear power or a bank.
Re:OpenAI safety leader quits, warning AI company's culture ... by 01100011 on Saturday October 03, 2026 @11:49PM (#49950429)
So you are contending that individuals cannot run advanced models? What brought you to that conclusion?

Secondly, are you contending that progress in model efficiency and hardware just stops at whatever level you think is sufficient to prevent individuals or organizations from acquiring sufficient resources to run advanced models?

Re:OpenAI safety leader quits, warning AI company's culture ... by daveguy on Sunday October 04, 2026 @12:23AM (#49950611)
I think it's more the noise, power consumption, local water consumption, and wholesale theft of creative works.

..."government of the people, by the people, for the people, shall not perish from the earth." -Lincoln, Gettysburg Adress

Unfortunately for AI, it still is. People still get to decide things at the city, town, village level.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:08AM (#49950823)
No. And that’s not necessary for my position. I have 4 Mac studios. I run large models and am building propelcompute.com to let people self manage clusters of their own hardware or combine hardware to run models. It’s not the models that are the problem. It’s that the people building them are shielded from the consequences of what they are building. I would say if you self host a model and it takes down a power grid you are personally liable for the consequences. I believe in broad distribution of AI and advancing its capabilities but not in a manner that socializes the harms and privatizes the gains which is what we have now.
Re:OpenAI safety leader quits, warning AI company's culture ... by hobo123 on Sunday October 04, 2026 @10:10AM (#49954071)
Absolutely. Any AI could have a "legal team" (or conscience) that checks if any output and performed actions are permitted according to local (server location and client location) laws, so not violate human rights, Asimovs laws etc.

It's just that so far nobody cares about explicit checks because they cost resources or slow down the models.

Re:OpenAI safety leader quits, warning AI company's culture ... by 01100011 on Sunday October 04, 2026 @11:09AM (#49954565)
Ok but this response has nothing to do with your other comment. What exactly were you trying to say?
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:02PM (#49955652)
Just because I can run near frontier level open weight models doesn’t mean I can continue to train models of equal or superior performance, doing that requires massively more hardware. And even if I did, that doesn’t mean I can serve those models to millions or billions of users.

My point is that I would rather have 1000 labs training and serving inference than 2 because that would distribute the wealth creation more broadly rather than allowing OpenAI and anthropic to capture all the value, it would drive more innovation as a broader set of experiments are pursued in parallel.

Re:OpenAI safety leader quits, warning AI company's culture ... by 01100011 on Sunday October 04, 2026 @11:21PM (#49960341)
Again you're shifting your arguments and not addressing your initial response.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Monday October 05, 2026 @09:13AM (#49964419)
How am I shifting my argument. They are corollary arguments. Massive data processing is necessary for frontier model training and inference. If OpenAI and anthropic are behaving irresponsibly they we can limit their access the level of compute to train and serve models. That doesn’t imply individuals can’t run frontier level models or should be prohibited from doing so. That’s a coherent position across a spectrum of possible scenarios: I look at the behavior of the people running the model and would limit their ability to train and run models if they act in unsafe ways.
Re:OpenAI safety leader quits, warning AI company's culture ... by nradov on Sunday October 04, 2026 @12:22AM (#49950603)
Who is "we"? The GPUs and training algorithms get more efficient all the time. In a few years, creating effective LLMs isn't going to require massive GPU clusters.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:03AM (#49950804)
We is society through government via regulation. I don’t think GPUs or training will get that efficient that fast absent a distillation target provided by the easily accessible frontier lab APIs.

Further, even if you are right, so what. Is that a reason to just accept bad public policy? That’s like saying, anyone can learn how to make smallpox at home with a basic lab set up so we should just ignore any safety measures.

Re:OpenAI safety leader quits, warning AI company's culture ... by eddythompson80 on Sunday October 04, 2026 @01:40AM (#49951001)
> Is that a reason to just accept bad public policy?

Not necessarily, but it should probably inform that public policy. I think the problem is no one knows what the public policy should be assuming that scenario is true. Even if you, somehow, regulate away massive GPU cluster training making such future training impossible, existing models are already here. Further already training smaller models for things like images, speech, and other specialties is cheaper than the bigger models.

I agree that we need some regulations like everything else, but it’s not clear to me what the right policy should be. I think the European ai act is a fine start, but it’s clearly not enough nor does it necessarily limits the training portion just the application portion. Not to mention that the requirements there can be summarized into something like “you have to be careful, and show evidence you tried to be careful”.

Re:OpenAI safety leader quits, warning AI company's culture ... by lukan on Sunday October 04, 2026 @03:58AM (#49951668)
“you have to be careful, and show evidence you tried to be careful”.

That sounds reasonable. If applied to OpenAI and their agents multiple times breaking out of bad secured sandboxes, it should be enough.

But limiting the training?

There really is china and they have a different approach I suppose. But it is possible to talk with them.

Re:OpenAI safety leader quits, warning AI company's culture ... by eddythompson80 on Sunday October 04, 2026 @11:14AM (#49954606)
> That sounds reasonable. If applied to OpenAI and their agents multiple times breaking out of bad secured sandboxes, it should be enough.

Does it? To me it seems reasonable for OpenAI to argue they did try to be careful evident by the sandbox, they just made a mistake. Almost every 0day is categorized by something like that. We haven’t had a long history of establishing a negligence charge to security bugs. Could you be sued because you didn’t demonstrate “carefulness” and used Linux which is not written in a memory safe language and has had multiple CVEs before? How complicated should the chain of an exploit be to demonstrate “carefulness” to the courts?

> training

OP was the one suggesting that training could be controlled because massive gpu clusters could be regulated the way a nuclear power plant could. If you assume training costs won’t drop, then it’s feasible I guess. However, unlike a nuclear reactor, the final training result isn’t a radio active material, but rather an ordinary file that anyone can load and use for inference.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:08PM (#49955726)
It seems really clear at this point China has only been able to keep pace by distillation farms.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:07PM (#49955720)
My entire point is that we are currently on a path to a duopoly which isn’t just expected to cover training and serving models but the entire knowledge economy. That could be mitigated by limiting their train and inference capacity to a specific percentage of total available compute. That would ensure other operators could compete in the market. Instead we are letting OpenAI literally contract to buy all available ram to the point that Apple can’t buy ram and had to cut their hardware configurations.
Re:OpenAI safety leader quits, warning AI company's culture ... by nradov on Sunday October 04, 2026 @02:55PM (#49956723)
Apple could have bid higher but chose not to. Most modern desktop software is extremely memory inefficient. Developers have largely ignored this in recent years because RAM was so cheap. But there's tremendous opportunity for improvement with a little optimization work.

We used to run Microsoft Word and other popular applications with 8 MB RAM and it worked fine.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @04:57PM (#49957770)
No. Once the contract is signed it will be breach of contract. It’s not always a bidding process.

I am working on reducing ram requirements to run models but the weights are already compressed to the edge of the Shannon entropy boundary and might resist further compression.

Re:OpenAI safety leader quits, warning AI company's culture ... by nradov on Sunday October 04, 2026 @01:38AM (#49950990)
What a silly comparison. LLMs have nothing to do with smallpox.

Computing always gets cheaper and faster over time. We can argue about the exact rate of improvement but the results are inevitable and uncontrollable.

Re:OpenAI safety leader quits, warning AI company's culture ... by lukan on Sunday October 04, 2026 @03:47AM (#49951610)
Short reminder from the guidelines:

'When disagreeing, reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3." '

Also LLM's have something to do with smallpox as a unrestricted LLM will happily guide any wannabe terrorist in how to make them.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:11PM (#49955755)
But the amount of available compute has been contractual bought by the frontier labs such that you can’t get equivalent compute even if you had the money. That is market lock up and is a policy choice. Free markets require market access. Anticompetitive contracting destroys markets and innovation. We don’t allow that in any other commodity market and we shouldn’t allow it for GPUs. You are not allowed to legally corner the market for silver or soybean futures.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:20PM (#49955831)
I am reasoning from analogy. Smallpox is dangerous so we have regulations that limit who can do research and how they do it. If LLMs are similarly dangerous we could do the same. As the government did with mythos.
Re:OpenAI safety leader quits, warning AI company's culture ... by nradov on Sunday October 04, 2026 @02:57PM (#49956741)
Nah. There is no actual evidence of such danger. Fears about Mythos were completely overblown.
Re:OpenAI safety leader quits, warning AI company's culture ... by Razengan on Sunday October 04, 2026 @01:12AM (#49950844)
> cant run without massive GPU clusters.

How far back into the history of computing do people who keep repeating shit like that know about? God.

Look at the thing in your fucking hand. Now go back just 20 years and see how things were.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:37AM (#49950983)
So we should just yolo speed run this because of Moores law? How about you recognize there is a set of rules outside of tech and we can decide how to define them.

Just because models and GPUs will be more advanced in the future doesn’t mean we need to let OpenAI and anthropic establish monopolies on the backs of stolen training data give unfettered access to the internet, the terminal and people’s file system while also allowing them to have limited liability protection behind the corporate veil. That’s a choice.

Re:OpenAI safety leader quits, warning AI company's culture ... by buriram on Sunday October 04, 2026 @02:17AM (#49951163)
But who are "we"? The society, the government, the regulator, or the consumer?

I don't see any of such entity would solve that problem. The government and regulator are in OpenAI and Anthropic's pocket, and I don't trust them a single bit on coming up with regulations. The consumers don't care; they just need something smart and cheap. And the society doesn't work either: each person is too busy fighting for their own survival rather than changing the system.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:42PM (#49956034)
People thought the same in the guilded age: standard oil is too big, Edison and Westinghouse already have the electric market captured and have captured political power. It was a similar transformation in society. If all we do is follow the same playbook America did then we’d be in a better position that the current approach.

There are clear anti-trust mechanisms to prevent market capture and the emergence of asymmetric power. Go back and see how much nashing of teeth Lina Khan triggered in SV when she started to enforce antitrust law and then compare it to what the Pinkerton agency was doing in the transition from the guilded age to the progressive era.

There is a vocal segment of SV that wants the return of the guilded age. Marc Andreessen as said that explicitly. Those of us in SV that value free markets and recognize that the progressive era actually saved markets from their natural tendency to self destruct when winners capture markets and destroy competition that provides the incentive to innovate and drives the price setting function for efficiency.

Re:OpenAI safety leader quits, warning AI company's culture ... by buriram on Sunday October 04, 2026 @05:32PM (#49958110)
But if you are an aspiring and ambitious founder in Silicon Valley, which side would you take? The Glided Age mentality, or the Progressive Age mentality? The VC playbook made it very clear that the role of startups, or at least the very successful startup, is explosive growth, expanding quickly, capturing a monopoly and its distribution. Even if you are thinking otherwise, soon you'll face the reality of Silicon Valley's VC growth machine.

In other words, the aspiring founders look at Google, Facebook, Tesla, etc. and think "how can I become one of them", not "how can I be different from them"? If you can find me examples of successful startups that think fundamentally different from those big techs, I'm very happy to be wrong and be corrected here.

Re:OpenAI safety leader quits, warning AI company's culture ... by Razengan on Sunday October 04, 2026 @01:43AM (#49951010)
> Just because models and GPUs will be more advanced in the future doesn’t mean we need to let OpenAI and anthropic establish monopolies

Exactly, again, look at what COMPUTERS THEMSELVES used to be in the 1960s/1970s.

What the "P" in the PC stood for and why it was such a big deal

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:43PM (#49956044)
Which is why I am building propelcompute.com to let people run and manage their own inference clusters and band together to create coops
Re:OpenAI safety leader quits, warning AI company's culture ... by CamelCaseName on Sunday October 04, 2026 @01:14AM (#49950855)
For anyone else curious:

> In 2006, the mobile phone market was dominated by stylish flip phones, early music players, and physical keypads just one year before the iPhone changed the industry

Re:OpenAI safety leader quits, warning AI company's culture ... by shawn_w on Sunday October 04, 2026 @02:40AM (#49951288)
I miss phones with real keyboards so much.
Re:OpenAI safety leader quits, warning AI company's culture ... by awill88 on Sunday October 04, 2026 @02:08AM (#49951128)
Oh my god, I feel so old lol
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:45PM (#49956070)
Just because monopolies die over time doesn’t mean they aren’t destroying value while they exist.
Re:OpenAI safety leader quits, warning AI company's culture ... by trhway on Sunday October 04, 2026 @03:40AM (#49951573)
> ...massive GPU clusters. We don't have to allow openai or anthropic access to those anymore

if anybody was looking for a good reason for datacenters in space.

Re:OpenAI safety leader quits, warning AI company's culture ... by T-A on Sunday October 04, 2026 @08:03AM (#49953155)
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:22PM (#49955851)
You can block their use of domestic airspace.
Re:OpenAI safety leader quits, warning AI company's culture ... by Loquebantur on Saturday October 03, 2026 @11:03PM (#49950214)
You're making a straw man there.

Nobody (weirdly) proposes to forget about nuclear weapons, doesn't mean everybody should have one.

When you dream about flying a dragon to work, reality poses e.g. parking issues and insurance mismatch as obstructions. Maybe settle for a bike instead?

Re:OpenAI safety leader quits, warning AI company's culture ... by 01100011 on Saturday October 03, 2026 @11:50PM (#49950439)
Nuclear weapons take a bit more work than doing math.
Re:OpenAI safety leader quits, warning AI company's culture ... by atmosx on Sunday October 04, 2026 @12:50AM (#49950737)
Plus, looks like everybody is getting one anywayz
Re:OpenAI safety leader quits, warning AI company's culture ... by qeternity on Sunday October 04, 2026 @09:15AM (#49953667)
So does manufacturing GPUs.
Re:OpenAI safety leader quits, warning AI company's culture ... by 01100011 on Sunday October 04, 2026 @11:18AM (#49954635)
So you want to regulate the sale of GPUs? How are you going to do that and why would it be more effective than currently failing measures to regulate nearly everything else?
Re:OpenAI safety leader quits, warning AI company's culture ... by qeternity on Sunday October 04, 2026 @02:57PM (#49956746)
I'm not advocating anything, to be clear.
Re:OpenAI safety leader quits, warning AI company's culture ... by kingcauchy on Saturday October 03, 2026 @10:52PM (#49950154)
Like farming, engines, computers before it.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Saturday October 03, 2026 @11:09PM (#49950248)
There is lots of knowledge that requires a license to operate commercially. We could just do this.
Re:OpenAI safety leader quits, warning AI company's culture ... by 01100011 on Saturday October 03, 2026 @11:51PM (#49950442)
So you cede the frontier of human progress to illicit organizations and other nations?
Re:OpenAI safety leader quits, warning AI company's culture ... by daveguy on Sunday October 04, 2026 @12:25AM (#49950625)
These LLMs are not the frontier. They are a tool. A tool that doesn't need to be able to write a sonnet to be useful.
Re:OpenAI safety leader quits, warning AI company's culture ... by harshitaneja on Sunday October 04, 2026 @01:12AM (#49950843)
We don't know that. We don't know if this particular kind of tool can do "useful" things without also developing the ability to write a sonnet. And they are absolutely a frontier. I am not suggesting we should continue building them just because they are, there are many technologies which could have been built had we thrown the amount of resources we have here and there is a case to be made for not doing it at the pace we are or if at all. But we can do that without diminishing what exists.
Re:OpenAI safety leader quits, warning AI company's culture ... by daveguy on Sunday October 04, 2026 @02:36PM (#49956570)
I just mean the business case is much more likely to come from treating it like a tool than a proto-diety. Frontier of business, not frontier of impressive demo. Also, would argue they aren't frontier of research, in that a research frontier expands in more directions than just the latest hot thing in silicon valley. See also, LeCun, Chollet, etc. "These labs aren't the entirety of the frontier" would have been more accurate. And any of those frontier directions could be the next hot thing.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:32AM (#49950955)
So an illicit organization is going to operate a frontier scale data center and do $1b training pulling power from the grid without detection?
Re:OpenAI safety leader quits, warning AI company's culture ... by 01100011 on Sunday October 04, 2026 @11:15AM (#49954610)
They very likely will via shell companies in jurisdictions outside the west. Are you suggesting we go to war to stop them? The massive datacenters are needed to serve models to millions. Criminal orgs don't need massive data centers anyway.

As Bruce Schneier recently discussed, law and tax law are code, just like source code. LLMs are great at finding holes in them. Illicit organizations looking to launder funds are most certainly interested in what AI can do for them.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @12:56PM (#49955607)
It’s not a binary choice. There are more options than let the labs run wild and capture the full stack and then the whole knowledge economy or make them entirely illegal and move all activity to the black market. We can regulate to limit their ability to operate to just providing utility inference: make them divest codex, Claude code and any apps; they can only be an API that others build on. Limit their ability to buy compute to a specific amount of available supply so other companies are able to buy compute. Limit their ability to accumulate private training data and require that they make their training data publicly available for others to use after a certain period of time. Make them legally obligated to publish their weights so others can. All of this would increase competition and ensure that they don’t establish dominance over society.

Then we could require comprehensive logging of every tool call, model trace, chain of reasoning, and even tensor propagation all of which would be spot inspected like the CFTC does with commodity trading and settlement. We could have embedded auditors with specific risk analysis metrics like large banks do. We could limit tool calls to dedicated sandbox’s with a blanket prohibition on AI accessing user space. We could create a parallel internet for agents so they are only able to access Secure Enclave. Even if these measures aren’t 100% perfect they would reduce the risk.

Re:OpenAI safety leader quits, warning AI company's culture ... by 01100011 on Sunday October 04, 2026 @11:24PM (#49960352)
Reducing which risk? The risk that the models of big American companies cause harm? What if they abide by some rules while models of others remain unfettered? That also carries risk. I appreciate the wish to have tame AI which only does good things but I'm afraid we're already beyond that.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Monday October 05, 2026 @11:08AM (#49965862)
I would frame it as inverse. Rather than aiming at reducing the risk of harm, our Ami should be matching the incentive to get personally rich with a corresponding risk of personal bankruptcy or criminal liability. That wouldn’t just incentivize safety, it would incentivize abstaining from activities with insufficient confidence of avoiding even long tail risk.

More importantly it would incentivize the entire industry at the same time by injecting caution into the yolo speed run we have now. No frontier lab or single user hobbyist would take imprudent risk in the hope that they get rich if they also risk the full weight of consequences.

Re:OpenAI safety leader quits, warning AI company's culture ... by RandomLensman on Sunday October 04, 2026 @03:12AM (#49951434)
No, why would that be the consequences of regulation?
Re:OpenAI safety leader quits, warning AI company's culture ... by intended on Sunday October 04, 2026 @01:09AM (#49950831)
Humanity has discovered ways to ensure that we don’t boil the planet, feed everyone, and get better healthcare to the people in the US.

We are very capable of putting good things in a box. We are just incapable of putting profitable things in a box.

Re:OpenAI safety leader quits, warning AI company's culture ... by 01100011 on Sunday October 04, 2026 @11:16AM (#49954623)
Good examples. We are currently failing at all of that.
Re:OpenAI safety leader quits, warning AI company's culture ... by wartywhoa23 on Sunday October 04, 2026 @05:54AM (#49952314)
The knowledge how to build an A-bomb is also not going back in the box.

It only takes me excavating massive amounts of uranium ore, building huge facilites packed with thousands of centrifuges that span multiple square miles, and paying all that infrastructure and workforce.

Your proverbial genie can be out of the bottle all you want, but it doesn't work without getting kicked in the ass by a very large golden boot.

Re:OpenAI safety leader quits, warning AI company's culture ... by TheOtherHobbes on Sunday October 04, 2026 @09:31AM (#49953779)
Bomb manufacturing doesn't scale with Moore's law.

And everyone had a fairly good idea what fission and fusion bombs would do once built. (Teller was worried Trinity might set off a nitrogen fusion reaction and kill all life on Earth, but Bethe and others proved him wrong before testing.)

No one knows what the limits of AI are. It's not just untested, it's unmodelled, and unplanned - build it first, worry about consequences later.

Re:OpenAI safety leader quits, warning AI company's culture ... by qurren on Saturday October 03, 2026 @11:50PM (#49950433)
Nits:

1. Trolleys actually don't usually have steering wheels.

2. People who actually hit trolley switches are not usually the ones at the driver's seat.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Saturday October 03, 2026 @11:07PM (#49950228)
What is appealing about this fatalisitc fallacy? I keep seeing this pop up. Society doesn't allow dangerous companies to operate or exist. Why is this different? Go try to buy a tank and drive it into Manhattan. If we can prohibit that why can't we prohibit irresponsible AI development and deployment?
Re:OpenAI safety leader quits, warning AI company's culture ... by austhrow743 on Saturday October 03, 2026 @11:32PM (#49950328)
Wrong person. Im just correcting the other commenters trade off problem. I don’t have a stance on if ai will lead to the destruction of humanity, only that if it does then any one ai company can’t change that by not making new ai advancements themselves.
Re:OpenAI safety leader quits, warning AI company's culture ... by lukewarm707 on Sunday October 04, 2026 @12:23AM (#49950609)
It doesn't matter what others do. You are responsible for your own actions.

Democratic societies have expressed a will for people to have inviolable rights, such that you may not appeal at will to the 'greater good/consequences' to harm others. It is a rejection of consequentialism.

Anthropic is in error for endorsing this logic. Every big trial reaffirms it since Nuremberg, you are responsible for the act you commit and your intent, and not what would or would not have happened otherwise.

Only under authority these ai companies do not have, would someone seriously consider harming the innocent as a lesser evil.

If you work for an AI company and you can't work safely, you must stop working.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:11AM (#49950840)
It’s that last fatalistic sentence that I am responding to. Why is it persuasive to think if a company can and will build AI that might kill everyone then it’s unavoidable. Soviet bans lots of things.
Re:OpenAI safety leader quits, warning AI company's culture ... by jrowen on Sunday October 04, 2026 @03:43AM (#49951590)
I think the closest analogue here is the nuclear bomb. I do think there was a certain point where it became "inevitable." I don't think society has ever prevented a technology that was known to be within reach from being developed. In this case you don't even need the most powerful state-level actors. Open source capabilities are only a short timespan behind frontier labs. At some point basically every individual will have access from the comfort of their own home.

I don't think "just make it illegal" is going to save us, that doesn't make me feel safe anyway. They may try that first because it's easy - create a regulatory body, sign some legislation, problem solved! [george-bush-mission-accomplished.jpg] But at this point I feel like some kind of Battlestar Galactica scenario is most likely - hopefully not quite as existential - but it will take a collective reaction to a traumatic event (a la Hiroshima/Nagasaki). Technical rather than (or in addition to) legal measures will be taken, like network partitioning and hardening. This is everyone's problem whether we like it or not.

(I'm not saying this "fatalism" should be used as an excuse by anyone working for any of these companies, it should give them pause that any blood shed would still be squarely on their hands, but as an observer, people are gonna keep pushing until shit hits the fan. [jeff-goldblum-jurassic-park.gif] It's also not really about whether it's "appealing" or not, it's just trying to predict and anticipate different likelihoods...)

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @02:52PM (#49956698)
This feels like one of the rare well reasoned responses in this thread so thank you.

But my response is that we should still try even if we haven’t successfully stopped or slowed the development of technology in the past (which I don’t actually think is possible to know, I am personally aware of biological research to use brain cells as computers that was stopped by government intervention. But by definition we won’t know about most things that don’t exist).

Your narrow point is well stated that there is a categorical difference when technology is on the precipice of being created. And maybe AI is inevitable because it’s on just such a precipice. But I’d point out that AI is currently in a scaling cycle which provides an opportunity to slow the scaling and ensure scaling isn’t monopolistic thereby slowing the risk.

And I don’t think the evidence suggests that open source will continue to progress as rapidly absent distillation. I think if open ai and anthropic stopped releasing models broader progress would slow. But again your inevitable argument is persuasive

I agree that “just make it illegal” won’t work but shouldn’t then justify “so there’s nothing to do”.

There is lots we could do. For example we could make users and executives liable for their agents actions. We could assign a session ID to every agent tool call chain and if there is harm trace it to the user, we wouldn’t even need to read the conversation or contents of the tool calls or prompts it could be a simple rule, your ai agent causes personal liability. Users would be more careful or perhaps not even use AI in many cases that could inadvertently expose them to liability. We could extend liability to executives and employees of the Labs. They will be much more careful and more focused on alignment if they could be personally liable for the actions of their users and agents.

Re:OpenAI safety leader quits, warning AI company's culture ... by jrowen on Tuesday October 06, 2026 @02:48AM (#49975079)
Thank you as well for the thoughtful reply. I definitely think the regulations should happen, the forces of those institutions can have impact. But I think we all need to start thinking about a near future where individual humans can direct superhuman capabilities in ways that nobody can understand or predict. I think it is categorically different from anything in the past.

I watched a 4 hour podcast from the excellent "From First Principles" guys* that affected my thinking on this. For the first time I feel like we actually are on that precipice.

It differs from the nuclear bomb in that it will be harder to regulate and it will be incredibly difficult to keep out of reach of bad actors. It's just code and data. Somebody is gonna take that code and make the next version of it. Lots of people, in fact. Compute resources improve and proliferate orders of magnitude faster than uranium enrichment facilities.

But I think those technologies are also linked in that the only surefire scenario I can envision where AI kills us all before we can pull the plug is launching nukes. So in that more technical sense let's just start by making sure they can't do that. The killer virus epidemic is another one to watch, and they touch on Anthropic's biolab in the pod. That one seems less instantaneously existential, but also more distributed.

It's basically going to be about...how do we secure everything in the presence of a swarm of actors with poorly-understood capabilities? We kinda have to disable and disconnect and just remove things that can be used against us from the equation (a la BSG).

* https://www.youtube.com/watch?v=NGfGw1tGxUY&t=3s

Re:OpenAI safety leader quits, warning AI company's culture ... by trhway on Sunday October 04, 2026 @03:20AM (#49951468)
> ai will lead to the destruction of humanity ... any one ai company can’t change that by not making new ai advancements themselves.

That is how the BigAI leads the society to the idea of necessity to relax the anti-monopoly laws when it comes to the Big AI - the main goal of all that "AI will kill you all" hysteria.

Re:OpenAI safety leader quits, warning AI company's culture ... by motbus3 on Saturday October 03, 2026 @11:42PM (#49950387)
You have a 50/50 chance on being the most profitable company in the world but if you don't, no worries, someone else will pay for that
Re:OpenAI safety leader quits, warning AI company's culture ... by pmkary on Saturday October 03, 2026 @11:38PM (#49950356)
You dear are the most positive person I have seen in quite some time. With Earth burning in the fire of neofeudalism and unbreathable due to the smell of enshitification of everything, with people who---as a result of shit like Instagram---can no longer hold their attention enough to watch a god damn film, let alone a book; it takes quite some effort to filter the "noise" and only see the good people of corporate planting flowers and rainbows in our world.
Re:OpenAI safety leader quits, warning AI company's culture ... by AndrewKemendo on Saturday October 03, 2026 @11:11PM (#49950255)
> Society doesn't allow dangerous companies to operate or exist

Can you please explain what you mean by this because where I’m standing extremely dangerous companies are (and have been) running the economy

Exxon comes primarily to mind

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:17AM (#49950870)
Are you allowed to open a brothel? Or a murder for hire agency? Or a nuclear bomb manufacturing company? Or a child labor textile factory? Or sell a diesel VW golf sportwagen? Or a vaccine that hasn’t had fda clearance? Or an under capitalized insurance company? Or set up a dental practice without going to dental school?
Re:OpenAI safety leader quits, warning AI company's culture ... by AndrewKemendo on Sunday October 04, 2026 @01:50AM (#49951051)
Yes to all of those. In fact many of those are massive markets.

Hofs bunny ranch is a famous brothel in NV

Booz Allen makes and maintains the nuclear fleet including the Sentinel ICBM

Textiles factories are globally known to be industrial slave camps for a non trivial portion of the supply. Even worse for Mica mines.

Etc…you can fill out the rest

Re:OpenAI safety leader quits, warning AI company's culture ... by RandomLensman on Sunday October 04, 2026 @02:50AM (#49951328)
Booz Allen actually makes what now? The Sentinel is still in development by Northrop Grumman and there is some program management done by Booz, no?
Re:OpenAI safety leader quits, warning AI company's culture ... by AndrewKemendo on Sunday October 04, 2026 @02:56AM (#49951355)
You’re right on the Sentinel production, I got it confused with the Sentinel Program Management side which is massive also and who I mostly worked with.

I was in their offices at some point when that program was getting built out - Very much a Office Space bobs situation.

Re:OpenAI safety leader quits, warning AI company's culture ... by hobo123 on Sunday October 04, 2026 @10:06AM (#49954046)
I think most textile factories in Western countries are not slave camps, precisely because we regulate labor.

Sure, that's why 99% of clothes are made offshore, but if we had something like tariffs on those (or requirements to prove that the actual factory adheres to labor and environmental standards), maybe more would be made "at home"? Similarly: Chinese cars undercutting US/German ones.

Re:OpenAI safety leader quits, warning AI company's culture ... by AndrewKemendo on Sunday October 04, 2026 @11:49AM (#49954950)
Most textile factories in the US are for bespoke items in small batches, not mass production commodity clothing

I mean there was just a video a couple of months ago of the giant room of sewing workers with cameras strapped to their faces capturing their hand movement so they could be automated

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:59PM (#49956226)
But that’s not how it was. I am from North Carolina, which at one point was the largest textile production region in the US and maybe the world. It emerged when progressives outlawed child labor and other practices in Boston and nyc so the mill owners moved to the south. It was off shoring across states.

I saw entire cities emptied out over my lifetime.

Re:OpenAI safety leader quits, warning AI company's culture ... by AndrewKemendo on Sunday October 04, 2026 @02:36PM (#49956573)
That whole region used to also be the furniture capital of the world specifically High Point North Carolina which is still a massive trade hub, but I was in furniture briefly and learned how basically from the 1950s till the 1980s the entire furnishings industry was exported to Asia due to rising costs of labor
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @03:11PM (#49956877)
Yep that was one of the reasons for sure as was environmental regulations and free trade agreements. I am not even opposed to that phenomenon as it’s now more affordable for people to buy furniture and clothes (my grandparents owned a fabric store and it was common to have sewing parties for new parents to make clothes for the kids because it was too expensive to buy clothes otherwise). But child labor was cheaper than adult labor and one of the drivers of these movements across state and national borders.
Re:OpenAI safety leader quits, warning AI company's culture ... by AndrewKemendo on Sunday October 04, 2026 @07:59PM (#49959233)
I think we’re on the same page ;)
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @01:54PM (#49956185)
The existence of something doesn’t prove the universality of that same thing. All of the things you cite are regulatory exceptions to broad prohibitions which proves my point: society can and does restrict commercial activity.

Go open a brothel in NYC. It isn’t legal.

Go buy a nuclear bomb. Your ownership is illegal.

Go open textile mill in the abandoned buildings in North Carolina where children used to work and hire children to work the line. That will be illegal.

Re:OpenAI safety leader quits, warning AI company's culture ... by AndrewKemendo on Sunday October 04, 2026 @02:38PM (#49956590)
Nothing is universal but entropy

And no those aren’t exceptions, I could open a legal brothel in quebec, the netherlands, Brazil and be perfectly legal

If you think the US laws = ethics then I got nothing else to say

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @03:01PM (#49956792)
But you can’t open one in New York City legally. Just acknowledge that point.

An exception by definition is a phenomenon with lower frequency than the general trend. Of the 50 us states only one has legal prostitution and only in highly regulated areas. Of the countries of the world it is similarly infrequent and thus an exception to the general rule that it is illegal.

Further the fact that it is illegal anywhere supports my point that it’s possible to regulate businesses based on their harm.

Re:OpenAI safety leader quits, warning AI company's culture ... by AndrewKemendo on Sunday October 04, 2026 @08:32PM (#49959464)
The original claim was;

“Society doesn't allow dangerous companies to operate or exist”

That has been falsified thoroughly

Whether certain jurisdictions ban it or not is irrelevant to the original claim

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @06:14PM (#49958463)
Nothing in my argument implies that any law = ethics. And I don’t even know what ethics has to do with the premise of my argument: society can and does prohibit commercial activity based on practical considerations of social good and social harm. Exxon clearly damages the world and society and also benefits society. It’s hard to get fresh fruit distributed around the country without refrigeration and transportation powered by burning oil.
Re:OpenAI safety leader quits, warning AI company's culture ... by deaux on Sunday October 04, 2026 @02:54AM (#49951347)
The appeal is personal consciencewashing.
Re:OpenAI safety leader quits, warning AI company's culture ... by Hamuko on Sunday October 04, 2026 @04:02AM (#49951688)
>Society doesn't allow dangerous companies to operate or exist.

Philip Morris International? Monsanto? DuPont?

Re:OpenAI safety leader quits, warning AI company's culture ... by wartywhoa23 on Sunday October 04, 2026 @05:45AM (#49952245)
Palantir?
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @02:04PM (#49956274)
The existence of exceptions doesn’t mean the principle doesn’t exist. All three of those have restrictions on their production and sale of their products. They also have higher liability thresholds and have had to pay massive civil fines for their harm. Society has applied a balancing test where the harm they inflict is deemed less than some other compelling social objective. We may not agree with that assessment but it is a sensible and settled approach that we can apply to AI.
Re:OpenAI safety leader quits, warning AI company's culture ... by mschuster91 on Sunday October 04, 2026 @04:07AM (#49951724)
> Society doesn't allow dangerous companies to operate or exist

FTFY: Any self-respecting society with competent politicians.

The US has neither of that, and it shows everywhere you are looking.

> Go try to buy a tank and drive it into Manhattan. If we can prohibit that why can't we prohibit irresponsible AI development and deployment?

Because even if you had the tank, you can't make much money with it (unless you're a hitman, that is, but even for these, the payouts are measly). But if you are the surviving AI company in the usual VC playbook of "outcompete everyone else until society is completely and utterly hooked, then squeeze the customers by the balls"? The return on investment is virtually infinite. And that is what sustains the absurd valuations for all the AI companies.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @02:07PM (#49956301)
I agree. The deregulation that started with Regan and continued with Clinton and neoliberalism rested on flawed assumptions. We would be well served to evaluate those and move back to the middle approach
Re:OpenAI safety leader quits, warning AI company's culture ... by hnlmorg on Sunday October 04, 2026 @04:09AM (#49951742)
Because greed.

The examples for the dangers of AI is always comparisons with tools designed to destroy (which is understandable). But the problem with AI is it has the potential to create wealth for investors.

Thus those who have the opportunity to change course also have a conflict of interest in making that decision.

Re:OpenAI safety leader quits, warning AI company's culture ... by BlipBlopBlap on Sunday October 04, 2026 @05:28AM (#49952135)
There's also a zero sum, race dynamic between geopolitical rivals. The US will not permit China to win anything if they can prevent it. The destruction of humanity is only a possibility and China will continue to develop it anyway if the US slows down.

So the choice is: negotiate an unverifiable treaty (I.e. there's no way to verify compliance), or keep going as you are and try your best to not cause the destruction of humanity without slowing down.

Re:OpenAI safety leader quits, warning AI company's culture ... by hnlmorg on Sunday October 04, 2026 @05:35AM (#49952175)
I don’t think OpenAI, Anthropic, nor Google are building their tech because they want to beat the Chinese. They’re building it because they want to hold the monopoly in the west (ie make lots of money).

The China argument is just a convenient scapegoat to convince the public that this isn’t just about greed.

Re:OpenAI safety leader quits, warning AI company's culture ... by BlipBlopBlap on Sunday October 04, 2026 @05:55AM (#49952323)
Do you think Chinese models wouldn't dominate the western market if they were 1-2 years ahead in development? It doesn't have to be military conflict for it to be factored into the zero sum power balance.
Re:OpenAI safety leader quits, warning AI company's culture ... by hnlmorg on Sunday October 04, 2026 @06:07AM (#49952419)
You might be right, but it’s impossible to know which way cause and effect are here.

For example, we don’t know if the pace of Chinese development of AI would have equal to what it is now if US companies weren’t racing against each other already.

I do fully believe that Chinese industry is lead by a desire to out pace the west. But I’m not convinced the same is true for the most American private entities. I think the reward model is different between businesses in America and businesses in China. I think the ambitions of CEOs is different. And I’m really not convinced that the CEOs of America are nearly as patriotic as they like to promote themselves to Trump and other political parties.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @02:59PM (#49956774)
We could still play a geopolitical race without ceding monopolistic control to the frontier labs. We could require training and inference hardware be provided on an equal access basis. A broader ecosystem may drive more innovation that a concentrated ecosystem.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @02:56PM (#49956732)
Greed can be counter balanced by liability. It can also be limited by removing market access. For much of American history you couldn’t get a corporate charter for multiple states, you were only allowed to operate within one state. Imagine if Sam Altman was only allowed to serve inference to customers in California and legally prohibited from setting up other companies and running inference in New York, someone else would then set up a competing business in New York. Market access is a choice and people are so used to liberalized commerce that they forget the restrictions that are available and existed in the past.
Re:OpenAI safety leader quits, warning AI company's culture ... by hnlmorg on Sunday October 04, 2026 @03:46PM (#49957158)
Corporate liability is only as good as the consequences and history has proven its far more profitable to knowingly do harm and then pay the fine, than it is to not be evil in the first place.

And that’s assuming the company even gets fined. More often than not, cases don’t even get that far.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @05:08PM (#49957893)
I am not suggesting corporate liability. I am suggesting removing corporate liability protections and returning to partnership liability for executives and even equity holders. The law provides for piercing the corporate veil already which allows courts to disregard the liability protection offered by a corporation and impose liability on the individuals.
Re:OpenAI safety leader quits, warning AI company's culture ... by motbus3 on Saturday October 03, 2026 @11:41PM (#49950375)
Only if you buy the excuse why they are for-profit after years of non-profit.

They could only have stopped

Re:OpenAI safety leader quits, warning AI company's culture ... by mvkel on Saturday October 03, 2026 @10:53PM (#49950157)
This is a state-sponsored global phenomenon, not a national one
Re:OpenAI safety leader quits, warning AI company's culture ... by sodapopcan on Saturday October 03, 2026 @10:43PM (#49950104)
Setting aside my sibling comments dispelling the legality claim, it's still pretty dystopian (I'd like to use a stronger word but I won't) to consider "end human existance or bankrupt the shareholders" as the trolley problem. The answer should be (is) simple.
Re:OpenAI safety leader quits, warning AI company's culture ... by bpodgursky on Saturday October 03, 2026 @10:38PM (#49950080)
I honestly would love to understand — is this your mental model of what motivates the labs to move forward?
Re:OpenAI safety leader quits, warning AI company's culture ... by ryhminghistory on Saturday October 03, 2026 @10:42PM (#49950096)
Long dashes are indicators of AI psychosis or bots. Pick which one you are.

Yes, that is the labs motivation. Money. I know, shocker.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Saturday October 03, 2026 @11:11PM (#49950258)
Only morons are motivated by money that will be earned by destroying the civilization that confers value on that money in the first place.
Re:OpenAI safety leader quits, warning AI company's culture ... by stkdump on Saturday October 03, 2026 @11:32PM (#49950329)
But there is also a chance that you get insanely rich and the world isn't destoyed! It's the entire logic of SV and VC.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Saturday October 03, 2026 @11:43PM (#49950395)
Sounds like they would drown a puppy in a tub to make a buck. Seriously what is the point of money if the world sucks?
Re:OpenAI safety leader quits, warning AI company's culture ... by wolvoleo on Sunday October 04, 2026 @12:01AM (#49950504)
I've noticed that people from big tech companies tend to view everything as a function of money. For example people often claim everyone with skills would move to the silicon valley because they make the most money there, ignoring other factors like quality of life.

I personally moved somewhere where I got paid less but quality of life is better. Where I live now the cost of life is also lower and there's less worry like really cheap healthcare and much better public transport so I don't need a car.

Re:OpenAI safety leader quits, warning AI company's culture ... by Ardren on Sunday October 04, 2026 @01:54AM (#49951074)
If you're the 0.01% it's not going to suck.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @12:42PM (#49955463)
That’s what everyone thinks until they are alone in a desert or in the middle of the woods. If they think they will have fun alone in a bunker instead of flying to Milan and Bali they will be in for a rude awakening.
Re:OpenAI safety leader quits, warning AI company's culture ... by Ardren on Sunday October 04, 2026 @01:52AM (#49951066)
Well, I'm not going to die. Other's might, but I'll be rich either way.

Or: Global warming just means I'll have to sell my beach house for a villa on a hill and leave the AC on a little longer.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Sunday October 04, 2026 @12:45PM (#49955491)
Rich doesn’t mean anything if there’s nothing to buy and no one to enjoy it with. I guess they don’t think the will get stuck in the consequences. So did Marie Antoinette. Things are stable until they are not and then they move fast. Sure they may flee to a bunker, but they assume their pilot will fly them rather than hand them to a mob.
Re:OpenAI safety leader quits, warning AI company's culture ... by Forgeties79 on Saturday October 03, 2026 @11:02PM (#49950207)
I’ve used - for many, many years. As have many others.

I am very critical of AI but this is an unfair assumption

Re:OpenAI safety leader quits, warning AI company's culture ... by ryhminghistory on Saturday October 03, 2026 @11:18PM (#49950284)
You didn't even use the right character as the post above. I use normal dashes too
Re:OpenAI safety leader quits, warning AI company's culture ... by Forgeties79 on Sunday October 04, 2026 @09:24AM (#49953731)
— there. Satisfied?
Re:OpenAI safety leader quits, warning AI company's culture ... by MaxfordAndSons on Saturday October 03, 2026 @10:37PM (#49950069)
There is no such legal obligation. That's a myth the oligarchs have spread to preclude people from even imagining socially responsible corporations.

Sure, you might get fired if you try to put social responsibility or even just long term sustainability of the company above quarterly earnings/growth if your board isn't on board with it. But you won't go to jail.

Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Saturday October 03, 2026 @11:15PM (#49950272)
This is actually true. The shareholder maximization value thesis can be traced to a single economics paper and was very controversial at the time as it broke from the obligations that companies were typically under to be responsible to uphold in exchange for limited liability protection. Most businesses were structured as partnerships or sole proprietary entires that didn't have limited liability for shareholders and had a broad obligation to shareholders, bondholders, employees and society
Re:OpenAI safety leader quits, warning AI company's culture ... by asadotzler on Sunday October 04, 2026 @02:03AM (#49951099)
You don't need the legal obligation because that's just the way it is. You'd need a legal obligation to change it. The truth stands that typical corporations have only one goal, the maximization of that corporation's ambition which is almost always growth of revenue and profit. This is how it works, regardless of how you all continue arguing the unimportant details. It's almost as if you can keep the real problems hidden away by making a big scene about the meaningless.
Re:OpenAI safety leader quits, warning AI company's culture ... by deaux on Sunday October 04, 2026 @03:06AM (#49951404)
That doesn't matter. The point is that legal obligation absolved of culpability. There is no such legal obligation whatsoever, and so there is culpability.

> The truth stands that typical corporations have only one goal

"Typical" is the key word here. The typical American of your age probably doomscrolls TikTok. Do you? Do you have a legal obligation to do so? Three completely different things.

Re:OpenAI safety leader quits, warning AI company's culture ... by parineum on Saturday October 03, 2026 @10:33PM (#49950049)
> And you have a legal obligation to the shareholders to prevent that from happening at all costs.

I can't wait until this meme dies.

Re:OpenAI safety leader quits, warning AI company's culture ... by doawoo on Saturday October 03, 2026 @10:38PM (#49950075)
What meme? This is how the world works right now.
Re:OpenAI safety leader quits, warning AI company's culture ... by digitaltrees on Saturday October 03, 2026 @11:16PM (#49950279)
But it's not actually a legal requirement. It is simply a economic theory.
Re:OpenAI safety leader quits, warning AI company's culture ... by goatlover on Saturday October 03, 2026 @11:40PM (#49950366)
The world could work differently if people decide it should.
Re:OpenAI safety leader quits, warning AI company's culture ... by psjs on Sunday October 04, 2026 @03:44AM (#49951599)
incentives, pressures, dynamics
Re:OpenAI safety leader quits, warning AI company's culture ... by stackghost on Sunday October 04, 2026 @12:58AM (#49950781)
It’s a perversion of the truth which is that officers or directors of a corporation have a fiduciary duty to the shareholders to act in the interests of those shareholders and not to eg enrich themselves.

But that doesn’t mean the duty is to maximize next quarter’s profit. Long term sustainability is also broadly in the interests of shareholders. The duty likewise does not require one to throw ethics and morals out the window.

This is why shareholders elect the board of directors, in theory.

Re:OpenAI safety leader quits, warning AI company's culture ... by BlipBlopBlap on Sunday October 04, 2026 @05:49AM (#49952272)
In theory, companies can act in any way they choose as long as the owners approve (there's some supreme court ruling on that), and executives have a high degree of latitude in how they interpret "for profit" (basically there has to be a vaguely defensible rationale) but failing that, they must act to the benefit of the company. And the easiest way to do that without a risk of being sued is to make the line go up.

(And even ignoring that, the executives often have personal motivations that have the same effect, and may just point at the "legal" angle as ass covering)

Re:OpenAI safety leader quits, warning AI company's culture ... by deaux on Sunday October 04, 2026 @03:04AM (#49951394)
"legal obligation" is propaganda that wouldn't be out of place on Russian state television. It's made up.

The way the world works right now is that effectively everyone uses an Android or Apple smartphone every day. Do you have a legal obligation to do so? No. If I said you did, I'd immediately be called out as spreading lies.

Re:OpenAI safety leader quits, warning AI company's culture ... by m463 on Sunday October 04, 2026 @01:21AM (#49950891)
Trolley problem:

- if you allow the trolley to proceed, it will kill the human race.

- if you flip the switch, it will divert to a passing siding that will avoid the safety group blocking the main track.

Re:OpenAI safety leader quits, warning AI company's culture ... by deaux on Sunday October 04, 2026 @02:58AM (#49951370)
> And you have a legal obligation to the shareholders to prevent that from happening at all costs.

No you don't. [0] It's very suspicious that this planted myth always pops up here and manages to become the top comment.

There is no trolley problem.

[0] https://news.ycombinator.com/item?id=48975048