There was a supply chain attack last year where a commonly-used VS Code plugin updated and in the background essentially figured out which agents you had available, and then prompted them to exfiltrate any credentials it could find. Not even prompt injection.

Even ore likely, someone boosts search results on a dependency name that looks legit and the agent installs the dependency.

So while any kind of supply chain update could install malware without credentials, I think it's pretty unfortunate that the malware could then use the agent to more powerfully work against you

Prompt injection is indeed a thing. You are not paranoid. Even more so if the LLM in question is not very smart (frontier models are better at not falling for prompt injection, but not immune).

I don't think OP literally meant the agent is running as root though. "| bash" shouldn't do anything more than the user the agent is running at can do. The bigger threat is an instruction to "install <such and such package> in the box". Then the agent could reach out to the container and do just that.

He created his own small skynet. Give it full control on home appliances then one day it goes .

> I mean... right?

Yes.

> Am I too paranoid?

No.

After the hugging face and related incidents I don't think anyone can give these things the run of a box - and the internet attached to it - with a clear conscience.

> isn't this thing one bad prompt injection away from exposing your network / hosting a worm / worse?

I gave grok access to one of the projects I’ve been coding for a while using variety of agents (mostly gemini tho). It immediately flagged bunch of security issues, including exposed .env file.

[delayed]

I don't think I'd allow an unsupervised instance to do it, but I have an agent sandbox running on its own physical server (RPi 5) that has access to root on most everything and a bunch of high-powered credentials too (all encrypted at rest). It's a Claude code instance that lives in a tmux session. All of the ssh keys it has access to are via an SSH agent on my laptop that require me to touch-to-auth. I am happy enough with the risk/convenience trade-off, especially as it's allowed me to setup a properly segmented home network, tune Falco, and sweep for machines that need upgrades etc.

I am frequently surprised at how many otherwise pretty savvy people aren't using a password manager, or aren't using locked-down sandboxes for their agents, paste credentials directly into their agents, or aren't running regular sweeps on all their devices for exposed credentials (including agent transcripts). I'm just starting to work on "scan all content on my laptop and emails for sensitive information" as a regular service, but I do want to get proper local inference setup for that first.

We're experimenting with Copilot Code these days and if you're logged into your az cli in any form of shell it can use that session with the same user privileges you have.

In terms of compliance in the EU in a microsoft heavy shop, this is the closest thing we get to a "safe" coding agent and it isn't safe at all. It can run cloud only, but then it kind of sucks. I think the cloud only stuff is going to need to work better or we're going to need dedicated machines to run agents on where the device itself has very limited access. In those contexts I'm not too worried about them. Well, at least not about them being a supplychain risk, but they'd still be a risk to others if they had internet access (and since they're run by us, we'd be liable.)

You shouldn't need to "experiment" to determine this. How do you not know where your keys are? If the agent can read them, then it'll use them. This is going to be a very interesting time for all these devs who seem to have no idea how anything works.

You are not too paranoid. There was a post just the other day about the Anthropic agent program sending people's emails to third parties. All the third parties had to do was require an email address to interact and the LLM would oblige because it had access to that information.