You can jail the actual pedophile in that scenario, you cannot jail an agent. The real question is whether the person who instructs the computer to do the thing that leads to hacking should be liable, or whether the company that made the tool that when used unsafely lead to hacking should be liable. Gun manufacturer's typically are shielded from liability. They also have a lot more disclaimers on their products though.

>You can jail the actual pedophile in that scenario, you cannot jail an agent.

How is this relevant? Should we just jail the closest person that caused a bad, just to prove some sort of point?

>The real question is whether the person who instructs the computer to do the thing that leads to hacking should be liable, or whether the company that made the tool that when used unsafely lead to hacking should be liable.

This totally ignores the civil vs criminal distinction. Even in the case of guns, no one seriously thinks that gun company CEOs should be sent to prison for murder.

Thats gonna depend on the state in the US, but you can definitely get charged for negligently failing to do background checks as someone operating a daycare center. Operating computers on public networks, you assume a level of responsibility to not wreak havoc, just like a daycare operator assumes the responsibility of vetting caretakers.

>but you can definitely get charged for negligently failing to do background checks as someone operating a daycare center.

And openai isn't putting AI agents on the open internet either. They vaguely tried to sandbox the agents. You could argue that openai didn't do enough, but hindsight is 20/20, and in the pedophile case, you can probably come up with all sorts of clues post-hoc that the daycare could have picked up on, but didn't.

> openai isn't putting AI agents on the open internet either.

They did? Not in the training run which led to the HuggingFace hack, but other runs had agents being trained on information retrieval tasks, like the RubyGems hack.

> You could argue that openai didn't do enough, but hindsight is 20/20

No, you can argue they didn’t do enough because they didn’t.

Air-gapping is a concept you learn in cybersecurity 101.

There’s no world in which this is just incompetence, the sandbox must have been designed to fail, or at least not designed to contain an agent with capabilities that they had been publicly saying it had.

>They did? Not in the training run which led to the HuggingFace hack, but other runs had agents being trained on information retrieval tasks, like the RubyGems hack.

The agents were sandboxed, but they escaped the sandbox by using a 0day in artifactory.

>Air-gapping is a concept you learn in cybersecurity 101.

And that's plainly untennable in cases where access to docs/packages are required. You're just proving the "hindsight is 20/20" point. Most hacks could conceivably be prevented through airgapping, so does that mean every entity who got hacked is criminally liable because they didn't airgap their systems?

This isn't limited to air gaps, you can make similar arguments for other security measures. For instance computing history is littered with kernel EoPs and hypervisor escapes. Does that mean anyone who uses a shared hosting environment and gets hacked as a result should be blamed because they didn't follow "cybersecurity 101" and running everything on bare metal?

A local mirror of the latest N versions of every package and docs of any package ecosystem they want is perfectly viable.

No, not anyone who uses a shared hosting environment, but certainly anyone who runs unaligned models they know have cyber weapon capabilities. They should be considered an untrusted 3rd party.

>A local mirror of the latest N versions of every package and docs of any package ecosystem they want is perfectly viable.

Something tells me that's harder than it sounds, given that even ci/cd environments don't bother maintaining package caches for npm/pypi, even though that would provide massive bandwidth savings.

They do, though. AWS offers CodeArtifact, which is basically that as a service.

That just proxies to the origin repository, creating the exact issue that artifactory has.

Of course. I was just challenging your point that no one is building an npm/pypi cache.

edit: This [1] 2022 dataset of npm packages had 19 TB, containing every package and every version. Perfectly viable for OpenAI, even if we estimate npm grew significantly in 4 years. In practice you wouldn’t need to have every version for it to be functional.

1: https://www.jonbell.net/preprint/msr23-npm.pdf