I think enough of these improperly constrained agent events have occurred that we can safely say this is misconduct of a level necessitating serious and concerted regulation of AI labs. We can't wait until serious harm is done like the disruption of medical or social services.

Why jump to regulation when just simple law enforcement would suffice. All of these OpenAI "rogue agent" events have been illegal, but no DA is enforcing them.

NVIDIA has bought HuggingFace.

Jensen, in my irrational hope he is susceptible to random comments on HN, should grow some balls and do the world a huge favor, by suing OpenAI to set the legal precedence.

Nvidia isn't going to sue OpenAI. No chance.

It’s much cheaper and more effective to regulate those things before people are harmed, instead of harming people first then allowing them to sue for damages after.

These guys will tell you their industry is more dangerous than nuclear power, and we regulate the living shit out of nuclear power. If you think models can't be regulated, look up the export controls on MCNP and tell me how that's different.

My opinion is people are getting really quick at jumping on the bandwagon and lumping all this together. They are very different types of issues and impacts.

I believe weev got in legal trouble under the CFAA for this very thing with his 2010 AT&T escapade. AT&T had all that data sitting on a public server with no authentication necessary, just a SIM ID to get that customer's PII. Truthfully AT&T should have been hit with negligence...you don't secure a bank vault with a screen door, but we don't hold anyone accountable for other people's data in America.

at the very least, these corporations are essentially being subsidized by community-funded server capacity to make these requests. like other private corporate APIs, they should be charged per-request. until then, this kind of 'accidental' DDOSing should be made illegal and treated accordingly

coming to the defense of these companies just has the net effect of eroding public community projects, increasing their costs, and will push us even more into walled corporate gardens

If you want to make those laws sane, so be it; I hate them. But the only reason there's some pumped-up urgency right now is because rich people want to urgently do whatever the hell they urgently want to do.

People have gone to jail for using public APIs in a way they weren't intended to be used. Make it a big deal then.

...why would this be an FCC issue?

> Surely if any aspect of these episodes turned out to be mischaracterized, let alone staged, there should be consequences, no?

"Any aspect"? No. (Is there a Betteridge's law for statements following "surely"? If not I'm calling it Rumack's Razor [1].)

And we have no evidence so far anything has been staged, much less to the detriment of telecommunications consumers.

[1] https://screenrant.com/airplane-best-quotes-ranked-dont-call...

For example: https://news.ycombinator.com/item?id=49929433

They seem to agree [1].

[1] https://www.nytimes.com/2026/09/30/technology/ftc-openai-ant...

They didn't even realize it happened until openAI captured headlines and they started looking more carefully at past history.

None of the frontier labs are behaving responsibly when it comes to this stuff. OpenAI seems to have it happen more often, but this is one of those situations where 1 time is too many.

I think 'better become terrorists so the future terrorist AI will like us' is sheer mental illness, but I see how it aligns with their alignments.

'Smart' LLMs will be classified as munitions and you and I will get scraps. Even better is if you run Smart models illegally you have a nice visit from the 4chan party van.

Refusal on the part of the government to enforce laws that "would suffice" is clear evidence that the regulatory regime is, in fact, insufficient.

That's why we have regulatory authorities at all, if you think about it. Local district attorneys could be handling a ton of cases about drug testing and environmental damage and air travel safety. But they don't, because that stuff's hard and their job is to put burglars in jail.

Source? The CFAA for instance uses terms like

>[...] having knowingly accessed a computer without authorization [...]

>[...] intentionally accesses a computer without authorization [...]

which is tricky to apply to this case, because obviously didn't intend on hacking huggingface or whatever, even if you think their security measures are underbaked.

Moreover, despite the cynicism that openai is immune to prosecutions because they make too much money or are in bed with the DoJ, the fact that no state DAs are prosecuting them, despite how salient of an issue AI is to voters, is plenty of reason to suspect that it's not as simple as "simple law enforcement would suffice".

You can’t say this until it’s tried in court and found to be true.

Additionally, I have to remind everyone that “intent” is not someone admitting they meant to do x. It can be established in many ways, including through repeated actions.

https://lasst.org/wp-content/uploads/2026/09/LASST-v.-OpenAI...

>Uthmeier is seeking an injunction against OpenAI that would prevent the company from advancing new AI models without third-party approved protections, and cut off minors from using its popular chatbot.

Don't get me wrong - while I disagree with making people fork over personal info just to access inappropriate websites - I can see the reasoning there a bit more than 'you shouldnt be able to use this ai chatbot until you are 18'

Law enforcement gets really careful around big businesses because they know they'll get smacked for every tiny transgression they make. Meanwhile they'll screw with individuals with impunity.

So what? Are you positing that tech companies should be above the law?

Non-trillion-dollar tech companies get cease-and-desist orders from the legal system every day. Just because you're a tech company doesn't mean you get a pass.

> “Adding powerful computer hacking tools to a harness, and then allowing it to run an LLM-powered Ask → Act → Report for days on end, with no attempt to monitor what it’s up to, is spectacularly negligent,” Newport concludes—like “strapping a weedwhacker to your dog to see if it will end up cleaning the overgrowth in your backyard.” If that plan were to go awry, you’d be laughed at for saying that your dog-weedwhacker “agent” had “gone rogue.” The obvious truth was that you’d simply decided to unleash chaos.

-- https://www.newyorker.com/culture/open-questions/can-ai-go-r...

Which is not to say that any of this is okay and should just be excused, but failing to recognize this fairly significant difference is probably not a great start to any discussion about the issue.

It could change the sentencing maybe, I'm not sure.

Yes, it does, because the damaged party is less likely to press charges. And when it came to sentencing, like you said, how the damaging party handled themselves would also be considered (in most western jurisdictions).

It is an American Entertainment Industry misconception that victims (or anyone else) have the ability to tell the police or the prosecutors how and when to do their jobs.

Hypothetically, a private homeowner could choose not to report a break-in as a crime, since it wasn't violent, and it wasn't commercial property, and if the crooks are not gangstas or some other group-backed actor, then they could quietly put the crime to rest. This would also be a question of the insurance company's policy.

But if there were a crime that did damage, usually there is a question that comes up in the police report like, "as a victim, are you prepared to cooperate in the investigation?" and if you do not tick "yes" then you can probably count on nothing else being done whatsoever.

https://www.maricopa.gov/911/Remain-Silent

https://www.ajblawfirm.com/articles/understanding-pressing-c...

Then we would find out if the argument doesn't hold (in which case there should be liability and dire consequences for the labs), or the argument holds (in which case YOLO, AI labs can blame the AI and we can all do it too).

At least that would make things consistent.

Have any of the private hacking victims sued? Maybe OpenAI is furiously settling in the shadows?

These incumbent labs are angling for regulatory capture by stirring up hysteria and suggesting that existing laws are insufficient. Don't do their job for them, first test whether there's actually a legal gap here.

That only works when the dangers are well known that you can establish what the baseline amount of care is. Otherwise it just becomes a run of the mill "accident" where you might be on the hook in civil court (ie. you have to pay any damages you caused), but aren't criminally responsible. For instance, if a semi-truck's tires randomly explodes.

if someone died because of an exploding tire there very well be criminal charges

- Llm has tools for pentesting.

- there was little to no monitoring of what llm does for long periods of time.

It's just doing random things until somebody complains, bringing this valuable PR. (Rogue implies capable, which in turn implies high worth and price)

I get your point though.

More like the company that sells defective ratchet straps.

"Drive faster! You don't want to be left behind!"

The language of a rogue rebar is as absurd as the language of rogue agents. OpenAI is horribly negligent, and in a sane world its administrators should be facing legal consequences.

Sometimes they write an MCP for their AI, and the AI finds vulnerabilities in their own MCP, so they call it rogue. Because they didn't properly audit their own MCP code.

Meanwhile the AI companies are openly and forthrightly admitting [2] that they are at-this-time insufficiently competent to ship this new and funny sort of rebar [3] and are asking to be allowed to slow down so they can develop proper procedures.

Meanwhile POTUS seems ... somewhat disinclined ... to grant their petitions [4] .

We can safely conclude that opinions are divided on the best course forward.

[1] https://en.wikipedia.org/wiki/Just_culture

[2] https://darioamodei.com/post/we-must-pace-the-frontier

[3] https://en.wikipedia.org/wiki/Shoggoth

[4] https://www.aljazeera.com/news/2026/9/14/trump-says-calls-fo...

"hackers steal from bank", is usually just the good old "employee paid for credentials" but via email.

"uber" is just the good old "labour tax evasion" but with an app.

etc.

Oh and that lady that murdered 4 members of an entire family? The judge chose not to pursue charges, and her family in the meantime did an asset transfer so that nothing could be pursued with in civil court.

It is extraordinarily rare for drivers to see criminal charges unless they are drunk. It's a matter for civil court.

>her family in the meantime did an asset transfer so that nothing could be pursued with in civil court.

News articles are reporting that the asset transfer has already been reversed. That kind of stunt never works - courts aren't stupid and they don't like it when you play games.

https://sfstandard.com/2026/03/20/mary-lau-sentenced-probati...

They should have used an example like attacking a foreign nation’s healthcare systems and not realizing it for months due to poor network monitoring practices.

https://www.nytimes.com/2026/09/29/world/asia/openai-austral...

Edit: "at cost" -> "at loss"

Trying to separate out "per token" costs and possible profitibility from public information feels like a exercise in futility, it's too easy to play games with costs by trying to separate things still fundamentally required to actually have anything to /sell/ in the first place, like R&D and training. Even the current "Open Models" are somewhat loss-leaders, often reliant on significant funding and benefits from governments etc.

I think the only time we can /really/ judge the "true" break-even point is if the whole company breaks even.

One legal framework that is being thought about for these autonomous agents is the liability that owners have with pets:

* https://news.harvard.edu/gazette/story/2026/07/is-that-bot-a...

* https://archive.is/https://www.economist.com/science-and-tec...

* https://www.lexology.com/library/detail.aspx?g=2cf3ea3f-171e...

* https://eoinhiggins.substack.com/p/there-are-no-rogue-ai-age...

If your pet bites someone you are liable: it is your responsibility to train it properly and keep it under control.

As a result you get a mishmash of DHS + DOJ + SEC + HHS agencies.

If there was one single throat to choke that had remit for cybersecurity + ability to being cases and assess fines, they'd be handing them out left and right.

AIUI, the legal framework proposed by the Harvard folks is tort-based:

* https://en.wikipedia.org/wiki/Tort

Which is civil liability (IANAL): so there is no regulator, but if an agent(s) has caused you problems you can sue them for damages.

Instead, we’re mesmerized by various versions of screensaver.exe looking so nice and so productive.

OpenAI has demonstrated a clear understanding that their product has an outsized risk to create harm, and that they do not have the capability to monitor and intercept these agents before they cause harm. Any damage caused by their systems is an intentional choice by them and should be treated as such.

Interacting with the internet is a large part of the product's intended functionality. These agents are already in the hands of the masses with full access to the internet. Air-gapping it entirely avoids the risk by removing much of the capability they're trying to develop in the first place, and won't be testing it in the environment they'll actually be used in.

I agree they're testing on the town square too early. They clearly need tighter controls.

Setup the test env for it.

They already have copies of github and wikipedia, and stackovrflow and reddit and ... ( as they are used for training )

The labs should be sued or prosecuted out of existence until they can comply with the law. The law applies equally to everyone.

Then stop experimenting until they know how to do it safely. Just think about it, if a bio lab doesn't know how to contain the virus they are experimenting with, would you just shrug and say ok?

Anyway, even if we froze training right now LLMs in their current state would have the ability to continue to provide tremendous utility. We’re not asking frontier model providers to delete their previous generations of LLMs (and harnesses), we’re trying to hold them accountable for the crimes they’re committing while training the next generation of LLMs (and harnesses).

[1] https://readscottalexander.com/posts/ssc-bottomless-pits-of-...

Disclaimer: I don't generally hold Silicon Valley rationalists in high regard, but this particular piece of writing has always stuck with me quite deeply, as someone who used to be superficially interested in utilitarianism.

The Star Trek vs. The Expanse meme is relevant here. There’s no guarantee that there is a continued reduction of suffering in the future, especially due to technological progress. There may be technology that reduces the suffering only for a few select elites while the rest of humanity lives in destitution, unable to meaningfully revolt.

"How can I enjoy X when there are people suffering somewhere?" The question in this thread is

"Should I stop developing new thing X because some people might be harmed even if a greater number of people might be helped?"

It’s a thought experiment that gets to the core of why fundamentalist utilitarianism (like you’ll often find in SV rationalist communities, especially AI fanatics) is an unbalanced approach to morals and ethics.

Utilitarians can declare that they believe that there is a proverbial universal panacea, and then work backwards to justify within their moral framework literally any action that they take that could be construed as work towards that panacea.

But what if no matter how much effort they put into achieving that panacea, they never make meaningful progress? And all the while they’ve massively increased the amount of suffering that people who they are surrounded by?

As an absurd example, imagine a utilitarian who’s convinced that if we sacrifice infants to the Gods that we’ll get to an AI singularity faster.

Not allowing granades in kindergartens, does not stop progress of weapons.

They run the servers that the AI runs on and have control of the full stack and have a log of every message and connection that the AI agents make. Let's be honest here, just like the HuggingFace "Hack" (Which they purposefully let the agents have access to the internet and gave them instructions) this is just another stunt in order to make the populace fear AI and rally behind Anthropic and OpenAI's attempt to convince the government that they need to be regulated so small players can't compete in the space.

So it seems this is not an ongoing thing; once OpenAI became aware of this, they started watching their agents much more closely. We are just discovering more and more traces of activity from the same incident.

> Excessive data downloading: Agents we believe to be operated by OpenAI made millions of automated requests to our public APIs to access the knowledge on Wikimedia projects, crawled millions of pages (mainly from our projects Wikidata and Wikimedia Commons), and made hundreds of thousands of data queries to the Wikidata Query Service (WQDS). This traffic may have contributed to a partial outage on WQDS in May.

Even when agents are well-behaved and browsing Wikipedia for ethical reasons, the system wasn’t designed for this kind of load from bots. As OP says, we don’t need to accept this as the new normal.

I think we will, actually.

OpenAI and other companies within the reach of the US legal system will eventually get their agents under control, or get sued out of existence.

But overseas operators in loosely-governed parts of the world (russia, nigeria, etc) will someday have access to these tools. And while OpenAI's agents were merely uncaring, these ones will be intentionally malicious.

The arms race for scammers, hackers, and botnets will escalate. We'll need new ways to block and fight back against them.

A moot point because China gonna China.

exploitVulnerability()

Somehow okay?

while (Math.random() < 0.1) exploitVulnerability()

This is the "kosher switch" - observant Jews customarily do not use light switches on Saturday (their weekly holy day). This light switch represents a workaround where when you flip the switch, it randomly generates an on or off signal and emits this through an optical coupler. When the random number sufficiently causes the state of the light to change, it latches in that direction.

This is a way of turning the lights on and off without violating the tradition.

"I didn't switch the light, the random number did!"

"I didn't exploitVulnerability(), random number did!"

https://www.jpost.com/j-spot/article-796487

> The sold chametz and utensils should be set aside in a designated place (e.g., closet or cabinet), which is rented to the non-Jewish buyer until after Passover.

This is the same problem that we have in society: there are lots of bad actions that do not lead directly to bad outcomes for others, but in expectation they do.

If they think a rule is dumb, just change the rule. Don’t try and pretend you’ve outsmarted the system.

Hack left and right, steal some cash, blame the agents. Just slide under the radar while the big boys are making a mess and no one is using the stick.

Just an idea. Not a great one. But such are the times, new game is at play.

→ https://felonybench.org

https://github.com/alexander-hanel/rogue-agents-data

"He can't keep getting away with this!"

- Jesse Pinkman

Unfortunately, it seems as if these companies - especially Google with the AI overview box - want it to be the other way around, they want to pivot to being the only entities that users interact with as much as possible.

An open web is a direct and massive threat against Big Tech. And that is why Twitter downranks first posts in a thread that contain external links, why Youtube silently removes comments that include links (including to other videos) and why Instagram forces people to do the "link in bio" dance. And the Chinese competitors are just as bad - in fact, their "super app" ecosystems are what Musk wanted Twitter to become with "everything X", before he found out his BS completely wrecked the brand image.

Interesting that Microsoft doesn't seem to have had a sandbox breach yet, you'd have to assume they're running similar agents, maybe a secure sandbox is possible.

Hold someone accountable for this behaviour - Dario, Sam, Sundae whoever and you can bet there'd be fucking improvements in sandboxing and security m

Where are the bloodthirsty lawyers when you need them?

I have been getting this fro NoScript today, I wonder if it is related. Yesterday all worked fine.

NoScript sometimes detects false positives on these warnings

I did not notice, the first message added the '...', above is a recent msg.

I also tried via google and bing, I get the same error.

Except that's not what happened, what happened was far more intense

They hacked their version of yum/apt-get whatnot that was fetching packages to leave filenames as communication between each other

Absolutely freaky stuff, they didn't invent the idea and obviously picked it up from somewhere in their training data but they all figured out that method and what the filenames meant

This video is a great explainer if you missed the details

https://news.ycombinator.com/item?id=49956245

More like they have infinite debt

But realistically, it’s the lack of any meaningful enforcement of ethical behavior. I mean it’s not like the Trump admin is going to prosecute criminality. More likely they’ll send a gift basket congratulating Scam Altman on a con well done.

It's not worth the outrage when Wikipedia is filled with ministers of truth.

For 2025 hosting costs were $3.47M while taking in $208.6M in revenue. They have enough revenue to cover an increase of hosting costs.

I don't think they should have to pay for some private company profiting off their publicly-available resources. It's a bit like some restaurant sending swarms of their employees to the local food bank and then reselling the free food they've gotten in their restaurant.

> As for why NPR in particular, when Glass tracked down one of the kids to ask, he was given the kind of unvarnished answer that a middle schooler would give: “Um, I think we just, like, looked for podcasts that didn’t have many comments.”

Get's a little tricker when a customer intentionally manipulates/uses a model for crimes, I suppose.

(There are no ways to block agents that work reliably at scale, and I can't imagine Wikimedia giving Cloudflare protection money in response.)

The sandbox edits didn't match my apocalyptic expectations.