Yep, this is essentially it: GitHub could provide a secure on-issue trigger here, but their defaults are extremely insecure (and may not be possible for them to fix, without a significant backwards compatibility break).

There's basically no reason for GitHub workflows to ever have any credentials by default; credentials should always be explicitly provisioned, and limited only to events that can be provenanced back to privileged actors (read: maintainers and similar). But GitHub Actions instead has this weird concept of "default-branch originated" events (like pull_request_target and issue_comment) that are significantly more privileged than they should be.

I agree but its only part of what is happening here. The larger issue is that with a LLM in the loop, you can't segment different access levels on operations. Jailbreaking seems to always be available. This can be overcome with good architecture I think but that doesn't seem to be happening yet.

IMO the core of the issue is the awful Github Actions Cache design. Look at the recommendations to avoid an attack by this extremely pernicious malware proof of concept: https://github.com/AdnaneKhan/Cacheract?tab=readme-ov-file#g.... How easy is it to mess this up when designing an action?

The LLM is a cute way to carry out this vulnerability, but in fact it's very easy to get code execution and poison a cache without LLMs, for example when executing code in the context of a unit test.

GHA in general just isn't designed to be secure. Instead of providing solid CI/CD primitives they have normalized letting CI run arbitrary unvetted 3rd-party code - and by nature of it being CD giving it privileged access keys.

It is genuinely a wonder that we haven't seen massive supply-chain compromises yet. Imagine what kind of horror you could do by compromising "actions/cache" and using CD credentials to pivot to everyone's AWS / GCP / Azure environments!

There is nothing weird with that; the origins of that workflows are on-site CI/CD tools where that is not a problem as both inputs and scripts are controlled by the org, and in that context

> But GitHub Actions instead has this weird concept of "default-branch originated" events (like pull_request_target and issue_comment) that are significantly more privileged than they should be.

That is just very convenient when setting up the workflow

They just didn't gave a shred of thought about how something open to public should look

> There is nothing weird with that; the origins of that workflows are on-site CI/CD tools

Well, it is pretty weird if you end up using it on a cloud based open platform where anyone can do anything. The history is not an argument for it not being weird, it is an argument against the judgement of whomever at Microsoft thought it'd be a good idea. I'm sure that person is now long gone in early retirement. It'd been great if developers weren't so hypnotized by the early brand of GitHub to see GitHub Actions for what it is, or namely, what it isn't.

This also compounds with npm's postinstall defaults. In this attack chain, the prompt injection triggers npm install on a fork, and postinstall scripts run with the user's full permissions without any audit prompt.

  So you end up with GHA's over-privileged credentials handing off to npm's over-privileged install hooks.

  I've started running --ignore-scripts by default and only whitelisting packages that genuinely need postinstall. It's a bit annoying, but the alternative is trusting   
  every transitive dependency not to do something during install.

There is nothing stopping Zapier from having a log4shell style vulnerability that exposes you to the same. The only difference is you're treating Zapier as a blackbox that you assume is secure, and any security issue is theirs and theirs alone. While with GHA you share that responsibility with GitHub. GitHub can screw up with a log4shell type exploit in how they handle the initial GHA scheduling too, but also you can have your own vulnerability in which ever arbitrary code you run to handle the trigger.

You can also do a lot more with GHA compared to waiting for Zapier to support your scenario. Plus most people I knew who used Zapier connected it to some Lambda or another webhook where they got the data from there and ran arbitrary code anyway.

No, the real problem is that people keep giving LLMs the ability to take nontrivial actions without explicit human verification - despite bulletproof input sanitization not having been invented yet!

Until we do so, every single form of input should be considered hostile. We've already seen LLMs run base64-encoded instructions[0], so even something as trivial as passing a list of commit shorthashes could be dangerous: someone could've encoded instructions in that, after all.

And all of that is before considering the possibility of a LLM going "rogue" and hallucinating needing to take actions it wasn't explicitly instructed to. I genuinely can't understand how people even for a second think it is a good idea to give a LLM access to production systems...

[0]: https://florian.github.io/base64/

> despite bulletproof input sanitization not having been invented yet!

I don’t think it can be.¹

¹ https://matthodges.com/posts/2025-08-26-music-to-break-model...

Interesting article you’ve linked. I’m not sure I agree, but it was a good read and food for thought in any case.

Work is still being done on how to bulletproof input “sanitization”. Research like [1] is what I love to discover, because it’s genuinely promising. If you can formally separate out the “decider” from the “parser” unit (in this case, by running two models), together with a small allowlisted set of tool calls, it might just be possible to get around the injection risks.

[1] Google DeepMind: Defeating Prompt Injections by Design. https://arxiv.org/abs/2503.18813

Sanitization isn’t enough. We need a way to separate code and data (not just to sanitize out instructions from data) that is deterministic. If there’s a “decide whether this input is code or data” model in the mix, you’ve already lost: that model can make a bad call, be influenced or tricked, and then you’re hosed.

At a fundamental level, having two contexts as suggested by some of the research in this area isn’t enough; errors or bad LLM judgement can still leak things back and forth between them. We need something like an SQL driver’s injection prevention: when you use it correctly, code/data confusion cannot occur since the two types of information are processed separately at the protocol level.

The linked article isn't describing a form of input sanitization, it's a complete separation between trusted and untrusted contexts. The trusted model has no access to untrusted input, and the untrusted model has no access to tools.

Simon Willison has a good explainer on CaMeL: https://simonwillison.net/2025/Apr/11/camel/

That’s still only as good as the ability of the trusted model to delineate instructions from data. The untrusted model will inevitably be compromised so as to pass bad data to the trusted model.

I have significant doubt that a P-LLM (as in the camel paper) operating a programming-language-like instruction set with “really good checks” is sufficient to avoid this issue. If it were, the P-LLM could be replaced with a deterministic tool call.

zizmor (https://github.com/zizmorcore/zizmor) and actionlint (runs shellcheck on run: | blocks) provide some bandaid. zizmor detects quite a few typical injection patterns like branch names and shellcheck enforces quoeting rules in the shell snippets