I too use it extensively. But they’re very, very capable models, and the command line contains a bunch of ways to exfiltrate data off your system if it wants to.
Yes, it was a legit safety issue and worth being aware of, but it’s not it was a general case. Red teamers worked hard to produce that result.
Was it a paper or something? Would you happen to remember the reference?
This is nowhere near the contortions red teams sometimes go through. They noted in general that overly emphasizing initiative was taken ... seriously.
I use Sonnet and Opus all the time through claude. But I don't generally use them with dangerously-skip-permissions on my main laptop.
Just yesterday my cursor agent made some changes to a live kubernetes cluster even over my specific instruction not to. I gave it kubectl to analyze and find the issues with a large Prometheud + AlertManager configuration, then switched windows to work on something else.
When I was back the MF was patching live resources to try and diagnose the issue.
In my own career, when I was a junior, I fucked up a prod database... which is why we generally don't give junior/associate people to much access to critical infra. Junior Engineers aren't "dangerous" but we just don't give them too much access/authority too soon.
Claude Code is actually way smarter than a junior engineer in my experience, but I wouldn't give it direct access to a prod database or servers, it's not needed.
My way of explaining that to people is to say that it's dangerous to do things like that.
If it is not dangerous to give them this access, why not grant it?
I have a cursor rule stating it should never make changes to clusters, and I have explicitly told it not to do anything behind my back.
I don't know what happened in the meantime, maybe it blew up its own context and "forgot" the basic rules, but when I got back it was running `kubectl patch` to try some changes and see if it works. Basically what a human - with the proper knowledge - would do.
Thing is: it worked. The MF found the templating issue that was breaking my Alertmanager by patching and comparing the logs. All by itself, however by going over an explicit rule I had given it a couple times.
So to summarize: it's useful as hell, but it's also dangerous as hell.
(Having said that, I'm just a kibitzer.)
Problem is: I also force it to run `kubectl --context somecontext`, as to avoid it using `kubectl config use-context` and pull a hug on me (if it switches the context and I miss it, I might then run commands against the wrong cluster by mistake). I have 60+ clusters so that's a major problem.
Then I'd need a way to allowlist `kubectl get --context`, `kubectl logs --context` and so on. A bit more painful, but hopefully a lot safer.
You (and many, many others) likely won't take this threat seriously until adversarial attacks become common. Right now, outside of security researcher proof of concepts, they're still vanishingly rare.
You ask why I'm obsessed with the danger? That's because I've been tracking prompt injection - and our total failure to find a robust solution for it - for three years now. I coined the name for it!
The only robust solution for it that I trust is effective sandboxing.
I share your worries on this topic.
I saw you experiment a lot with python. Do you have a python-focused sandboxed devcontainer setup for Claude Code / Codex you want to share? Or even a full stack setup?
Claude's devcontainer setup (https://github.com/anthropics/claude-code/tree/main/.devcont...) is focused on JS with npm.
I wrote a bit about that in a new post this morning, but I'm still looking for an ideal solution: https://simonwillison.net/2025/Sep/30/designing-agentic-loop...
-create a separate linux user, put it in an 'appshare' group, set its umask to 002 (default rwxrwxr.x)
-optional: setup some symlinks from its home dir to mine such as various ~/.config/... so it can use my installed packages and opencode config, etc. I have the option to give it limited write access with chgrp to appshare and chmod g+w (e.g. julia's cache)
-optional: setup firewall rules
-if it only needs read-only access to my git history it can work in a git worktree. I can then make git commits with my user account from the worktree. Or I can chgrp/chown my main working copy. Otherwise it needs a separate checkout
I actually preferred running stuff in containers to keep my personal system clean anyway so I like this better than letting claude use my laptop. I'm working on hosting devcontainer claude code in kubernetes too so I dont need my laptop at all.
I feel this is overly exagerated here.
There is more issues that are currently getting leverage to hack with vscode extension than AI prompt injection, that require a VERY VERY complex chain of attack to get some leaks.
But that's a very big if. I've seen Claude Code attempt to debug a JavaScript issue by running curl against the jsdelivr URL for a dependency it's using. A supply chain attack against NPM (and those aren't exactly rare these days) could add comments to code like that which could trigger attacks.
Ever run Claude Code in a folder that has a downloaded PDF from somewhere? There are a ton of tricks for hiding invisible malicious instructions in PDFs.
I run Claude Code and Codex CLI in YOLO mode sometimes despite this risk because I'm basically crossing my fingers that a malicious attack won't slip in, but I know that's a bad idea and that at some point in the future these attacks will be common enough for the risk to no longer be worth it.
Again you likely use vscode. Are you checking each extension you download? There is already a lot of reported attacks using vscode.
A lot of noise over MCP or tools hypothetical attacks. The attack surface is very narrow, vs what we already run before reaching Claude Code.
Yes Claude Code use curl and I find it quite annoying we can't shut the internal tools to replace them with MCP's that have filters, for better logging & ability to proxy/block action with more in depth analysis.
Maybe it will never happen? I find that extremely unlikely though. I think the reason it hasn't happened yet is that widespread use of agentic coding tools only really took off this year (Claude Code was born in February).
I expect there's going to be a nasty shock to the programming community at some point once bad actors figure out how easy it is to steal important credentials by seeding different sources with well crafted malicious attacks.
Lots of ways his could happen. To name two: Third-party software dependencies, HTTP requests for documentation (if your agent queries the Internet for information).
If you don't believe me, setup a MITM proxy to watch network requests and ask your AI agent to implement PASETO in your favorite programming language, and see if it queries https://github.com/paseto-standard/paseto-spec at all.
More seen as buzz article about how it could happen. This is very complicated to exploit vs classic supply chains and very narrow!
????
What does "This" refer to in your first sentence?
The researcher has gotten actual shells on oai machines before via prompt injection
https://gitlab.com/txlab/ai/sandcastle/
Check it out if you're experimental - but probably better in a few weeks when it's more stable.
Nice job for coining the name for something but it’s irrelevant here.
How is someone going to prompt inject my local code repo? I’m not scraping random websites to generate code.
This sort of baseless fear mongering doesn’t help the wider ai community.
See comment here for more: https://news.ycombinator.com/item?id=45427324
You may think you're not going to be exposed to malicious instructions, but there are so many ways bad instructions might make it into your context.
The fact that you're aware of this is the thing that helps keep you safe!
And yes, these are all "skill issues" - as in, if they had known better this wouldn't have happened to them, however I think it's fair to call these possibilities out to counter balance the AI is amazing and everyone should use it for everything type narratives as to instil at least a little caution.
i.e. quite dangerous, but people do it anyway
You know what neighbors of serial killers say to the news cameras right?
"He was always so quiet and polite. Never caused any issues"
I’ve been using Claude code since launch, must have used it for 1000 hours or more by now, and it’s never done anything I didn’t want it to do.
Why would I run it in a sandbox? It writes code for me and occasionally runs a build and tests.
I’m not sure why you’re so fixated on the “danger”, when you use these things all the time you end up realizing that the safety aspect is really nowhere near as bad as the “AI doomers” seem to make out.