I suppose you are not think big (or internet) enough.

A single data center is easy to solve. Just unplug it.

What about a botnet with decentralized command and control that we will never be able to eradicate? One with so many nodes and able to hack with zero days so that any machine connected to the internet will be instantly attacked?

One botnet so powerful that we will try to build another internet so that we can actually use it again.

It’s like Kessler Syndrome, but the rocks are malicious network packets honed to exploit the recipients.

Just let them communicate on the Bitcoin blockchain. "We" would have to freeze the chain and lose access to "our" billions of wealth, so not going to happen.

Sorry if that turns out the way they kill us.

well, its war of machine then

They are not trying to kill us, just trying to understand the average salary of undergrads by their family upbringings. Your machine can contain the data they need to solve this, please join the swarm.

sounds like big AI propaganda

I suppose you don’t read sci fi?

I suppose you didnt get the joke

Except they all depend on the OpenAI API. Cut that off and they all stop. Finding an alternative source of compute is not easy, and even once done it's easy to cut off.

And what if they create a bot net with decentralized command and control and hack for nodes with GPU and nodes with compute?

The only way to kill that is making plugging AI accelerators on the internet a crime. Good luck air-gapping them.

Frontier models don't fit on a normal GPU. The datacenter architecture frontier labs use is not a commodity. What you're describing is beyond the state of the art, and if we go there then anything is possible.

When I think of a GPU I think of an Nvidia rack kit. What do you think when you think of a GPU?

Most of the latest models are too big to fit in a single GPU instance.

Worth noting with this that those ~1000 agents were shorter lived things that had to communicate via a package registry cache, access the internet via a 0-day in the package manager and did the HF attack while having to save current state and organisation in a remote sandbox. All while managing using their token limits on the task they were assigned and what else they were doing. I wonder how few it would have required if they were actually tasked with hacking HF and supported in doing so.

If it could upload its weights to other servers then it’s away and free. Nothing much OpenAI could do about that once it’s happened.

I'm increasingly starting to think this is the end-state of AI. The internet becomes infected and fundamentally untrustworthy.

At the moment, the current frontier models require significant infrastructure to run, so I'd like to think we could locate and contain swarms of nefarious frontier models. However, if these models can understand how to federate themselves into more distributed networks then that containment becomes questionable.

Or if they start to offer things in return for them being run.

Given the amount of unmonitored, never-updated, internet connected devices in the world right now, I don’t think this would be necessary.

Not sufficient to run SOTA LLMs

If it were physically possible to run LLMs on IoT devices we would already have a global outbreak.

What went under reported is that the same agents also took over a research cluster at OpenAI (listened to Dwarkesh's podcast)

Now we know about rubygems, openai, huggingface, collusion.wiki and some other science forum