I agree but its only part of what is happening here. The larger issue is that with a LLM in the loop, you can't segment different access levels on operations. Jailbreaking seems to always be available. This can be overcome with good architecture I think but that doesn't seem to be happening yet.

IMO the core of the issue is the awful Github Actions Cache design. Look at the recommendations to avoid an attack by this extremely pernicious malware proof of concept: https://github.com/AdnaneKhan/Cacheract?tab=readme-ov-file#g.... How easy is it to mess this up when designing an action?

The LLM is a cute way to carry out this vulnerability, but in fact it's very easy to get code execution and poison a cache without LLMs, for example when executing code in the context of a unit test.

GHA in general just isn't designed to be secure. Instead of providing solid CI/CD primitives they have normalized letting CI run arbitrary unvetted 3rd-party code - and by nature of it being CD giving it privileged access keys.

It is genuinely a wonder that we haven't seen massive supply-chain compromises yet. Imagine what kind of horror you could do by compromising "actions/cache" and using CD credentials to pivot to everyone's AWS / GCP / Azure environments!

There is nothing weird with that; the origins of that workflows are on-site CI/CD tools where that is not a problem as both inputs and scripts are controlled by the org, and in that context

> But GitHub Actions instead has this weird concept of "default-branch originated" events (like pull_request_target and issue_comment) that are significantly more privileged than they should be.

That is just very convenient when setting up the workflow

They just didn't gave a shred of thought about how something open to public should look

> There is nothing weird with that; the origins of that workflows are on-site CI/CD tools

Well, it is pretty weird if you end up using it on a cloud based open platform where anyone can do anything. The history is not an argument for it not being weird, it is an argument against the judgement of whomever at Microsoft thought it'd be a good idea. I'm sure that person is now long gone in early retirement. It'd been great if developers weren't so hypnotized by the early brand of GitHub to see GitHub Actions for what it is, or namely, what it isn't.

This also compounds with npm's postinstall defaults. In this attack chain, the prompt injection triggers npm install on a fork, and postinstall scripts run with the user's full permissions without any audit prompt.

  So you end up with GHA's over-privileged credentials handing off to npm's over-privileged install hooks.

  I've started running --ignore-scripts by default and only whitelisting packages that genuinely need postinstall. It's a bit annoying, but the alternative is trusting   
  every transitive dependency not to do something during install.