Unfortunately, it's not an easy problem to automate the detection. There are some heuristics, but you're basically writing something like Norton Antivirus for Python packages lol.
A sufficiently skilled attacker can make their code look totally innocent[0]. :)
Source: Former security engineer at Uber/Snap that's now working on tooling in this space[1].
0: "Underhanded C Competition" http://www.underhanded-c.org/
1: Open Source "Supply Chain Security" platform https://github.com/lunasec-io/lunasec
The only company that has a real chance at making a dent in the issue is GitHub and to a degree they already have.
SOURCES:
- https://www.mandiant.com/resources/blog/supply-chain-node-js
- https://www.zdnet.com/article/backdoor-found-in-ruby-library...
- https://www.reversinglabs.com/blog/mining-for-malicious-ruby...
- https://news.sophos.com/en-us/2021/10/24/node-poisoning-hija...
- https://www.bleepingcomputer.com/news/security/dev-corrupts-...
- https://portswigger.net/daily-swig/amp/malicious-python-libr...
Npm packages have really crazy dependency graphs. It's usually on the 1000s of times larger than what you get on most languages, 100s of times larger than python or ruby.
Hence left-pad. And this pet-peeve of mine https://nodejs.org/docs/latest/api/process.html#processargv (who cares about the node fullpath in argv[0]?)
The problem with left pad was a security one, not a package bloat one.
> It is highly probable that the majority of current string padding implementations are inefficient. Bringing this into the platform will improve performance of the web, and developer productivity as they no longer have to implement these common functions.
“It’s too trivial” is not why left-pad was in the news. Go read about it to understand the actual issue: https://www.theregister.com/AMP/2017/07/12/javascript_spec_s...
Of course it was useful, hence why most non-crappy languages had it in its stdlib from inception pretty much
But building a package just to do left-pad is stupid, especially since it can be implemented in a couple of lines
Nobody thinks leftPad was not a useful function. The question is, was it useful enough to counter all the risks of npm, probably not. In the stdlib there is no such risk.
My point has been this whole time that left-pad was not a story of a trivial function needlessly pulled from an external source as the person I replied to had claimed, and it appears you agree. Good!
Why on earth did it take until 2017 to add a trivial thing that nearly everybody needs at some point? It creates a need for super popular 3rd party “packages” just to do some basic data structure manipulation.
But, with npm, suddenly it was trivial to include other packages without having to worry about sub-dependencies. The whole tree just magically added itself to your build, and only if you paid attention to the build process would you discover just how many developers you were transitively trusting.
Programs in other languages with the same kind of tooling tend to stop at hundreds or at most low-thousands of dependencies. Javascript code often reach tens or low-hundreds of thousands of dependencies.
Dependency explosion is bad all around, but JS is exceptionally bad.
https://github.com/rust-lang/docs.rs
Follow the build instructions, then you get to `cargo build` you'll see this message
Downloaded 448 crates (44.1 MB)
react@15 — 20 deps
react@16 — 6 deps
react@17 — 4 deps
react@18 — 3 deps
If you pick your packages carefully (for example, looking for alternatives this one ↓ instead of the much more popular react-router), it's not as bad as it used to be.
A high level python package is more likely to depend upon 3-30 well known packages.
There is also the "Arch User Repository" which includes a bunch of useful packages that are easy to install, but with full transparency about where the contents are coming from (you can inspect the script).
They have even released an easy installed for Arch now that you can use, or you can just use Manjaro Linux. Regardless though, I really appreciate this distro for development after many years of using Ubuntu!
It's called archinstall and it's a helper script (it looks like)
The problem with piping is that bash will read line by line, keeping the connection open while it runs. If the connection fails for any reason, the script will stop, potentially breaking or corrupting things.
This can be prevented by first downloading the file, and then running it.
If the script is meant to be piped from curl, and is well written, it will be written so that it first defines everything as functions and then at the very end makes a single function call. This ensures that the script will only do anything if it has been completely downloaded.
For example, the script that rustup.rs tells you to pipe from curl is written in that way.
This is not sufficient. More care is required.
lsp_init() {
...
}
lsp_init
In other words, not only do you have to wrap your script in functions, but you have to ensure that any top-level function / command invocations are named such that they do not become different commands if truncated.
This is unsolvable in general, because the user can have any arbitrary names in their $PATH , such as custom personal utils in ~/.local/bin which can have any unforeseeable name.
It's much easier to just wrap the script in `()` to make it run in a subshell. bash will not run anything until it sees the closing `)` so truncated scripts are not a problem, and it also doesn't have the name collision problem.
For you, apparently. Other people do worry about it, which is why they do take care to wrap their script in functions. And my point is there's an even easier and more foolproof way than that.
>in the same way that a cosmic ray bitflip could cause a kernel bug that erases my entire drive but in reality I don't spend any time worrying about this.
Other people use ECC memory because they worry about this, and because it does measurably happen.
My shell[0] dies with a error "source file did not end with a newline". This is easily solvable in general, regardless of what's in PATH, as long as the shell works correctly. Valid text files do not end with bytes other than 0x0A.
0: That I wrote, because of bugs other than this one in bash et al, such as $FILE_WITH_SPACES.
Powershell will accept codesigning certs that are signed by verisign, so the workaround for an attacker who has already compromised a web site is to modify and re-sign the script with a certificate that can be obtained for $60.
Nope. Of course if someone stores a Code Signing key on the same server as the distribution server then all bets are off, but otherwise it's no possible.
PS CodeSign isn't perfect, but there is enough hoops to jump around for this not to be an easy task for a malicious actor gained access a distribution point.
Even on windows now, package are often signed by individual developers rather than an entity that you might recognise the name of.
> but there is enough hoops to jump around for this not to be an easy task for a malicious actor gained access a distribution point.
If this is an optional step (which it would have to be) then the easiest thing to do is to remove the signature from the script and modify the instructions on how to run the script. If an attacker has compromised the supply chain, unless the original host has done something incredibly dumb, chances are they're quite dedicated and jumping through a handful of extra steps arent going to stop them.
> Even on windows now, package are often signed by individual developers rather than an entity that you might recognise the name of.
This is implies what you, as an attacker:
have the access to the Enterprise PKI
can issue code signing certificates
somehow your certificate drops in the Trusted Publishers[0] on the endpoint ie you have the access or to the GPO/domain controllers (keys to the kingdom) or to the endpoint (making the whole dance with certs moot)
> If this is an optional step (which it would have to be) then the easiest thing to do is to remove the signature from the script
This would make the script to fail from running in a proper configuration
> and modify the instructions on how to run the script.
And again this requires the access to the endpoint, which makes your assumptions moot.
> If an attacker has compromised the supply chain, unless the original host has done something incredibly dumb, chances are they're quite dedicated and jumping through a handful of extra steps arent going to stop them.
Exactly, which is why you don't "signed by individual developers", don't provide the easy way to access the PKI, don't sign the scripts and yet run them with -bypass.
Come on, I did deploy PKI and PS Code Signing.
[0] https://old.reddit.com/r/PowerShell/comments/kpwi5r/powershe...
You still need to get that signature from somewhere, likely a web server. If your threat model is that you don't trust web servers but do trust the packages and repositories they give to you, then I guess this is true but that seems a little crazy given that the attacker can just change the website you're visiting to host a malicious package, and change the signature it verifies against.
> Of course, that signing key could also be compromised, but those are usually more tightly guarded than web servers,
Why is it reasonable to assume that? I don't think it is, at all. If their SSL certs are comprised, all bets are off.
> any reputable dev should have their key in a hardware enclave like yubikey or similar).
I suspect you will be very very very disappointed at the number of people who publish software that meet the criteria you've outlaid here.
Fundamentally, your argument is flawed right at the beginning because all the attacker needs to do is compromise the web server and change the installation instructions.
I have no statistics to offer, but it does seem likely to me that TLS private keys gets compromised more frequently than software/package signing keys.
Repository packages usually are signed with GPG. You can even use HTTP - it does not matter. And GPG could be used to sign packages on another machine, may be even on offline machine with HSM.
Things like pip or curl rely only on HTTPS. There are many agents between developer who built a deliverable artifact and your machine. There's WWW-server. There's some CDN with HTTPS termination. There's NSA which can issue fake certificate and MITM attack you (mostly imaginably threat for sure, but still).
And hacked WWW servers already happened. Transmission website was hacked and their binary was infected.
Now I don't claim it to be a big issue. Malevolent developer is much more dangerous and much more likely to happen. But if we want to compare those two methods - I think that repositories with proper signatures are safer.
curl -o file.sh
bash file.sh
Unless there are other sane ways to install the project, I just put in on my blacklist and ignore it.
A platform like PyPI also increases chances of something like that getting detected after it happens, giving you a chance to react in a timely way.
I almost always decide whether to execute someone’s else’s software based on their reputability. Is the project well known? Is it maintained, with lots of GitHub stars? My decision is never based on reading the source code.
I don’t have nearly as much of a problem with curling shell scripts as the finger-wagging “actually you shouldn’t do that” crowd seem to say I should.
The one thing that really does give me the creeps is adding someone else’s apt repository to my system wide one. I just can’t seem to square that off in my head against my more laissez-afire “well do I trust them or not?” attitude that I take for every other installation attack vector. Maybe because it means trusting the third party forever, instead of just once?