I was really nervous when "language package managers" started to catch on. I work in the systems programming world, not the web world, so for the past decade, I looked from a distance at stuff like pip and npm and whatever with kind of a questionable side-eye. But when I did a Rust project and saw how trivially easy it was to pull in dozens of completely un-reviewed dependencies from the Internet with Cargo via a single line in a config file, I knew we were in for a bad time. Sure enough. This is a bad direction, and we need to turn back now. (We won't. There is no such thing as computer security.)
I was trying to build just (the task runner) on Debian 12 and it was impossible. It kept complaining about rust version, then some libraries shenanigans. It is way easier to build Emacs and ffmpeg.
But... We absolutely are.
Now you get npm whining at you about an unsatisfiable dependency cycle because some three-level-removed transitive dependency you've never heard of put a hard lock file in that reference a version that got pulled for a security flaw.
On top of that, I try to keep the dependencies to an absolute minimum. In my current project it's 15 dependencies, including the sub-dependencies.
[1] https://github.com/ValveSoftware/Proton/commit/f21922d970888...
The distro package manager delivers applications (like Firefox) and a coherent set of libraries needed to run those applications.
Most distro package managers (except Nix and its kin) don't allow you to install multiple versions of a library, have libs with different compile time options enabled (or they need separate packages for that). Once you need a different version of some library than, say, Firefox does, you're out of luck.
A language package manager by contrast delivers your dependency graph, pinned to certain versions you control, to build your application. It can install many different versions of a lib, possibly even link them in the same application.
> Most distro package managers (except Nix and its kin) don't allow you to install multiple versions of a library
They do, but most distro only supports one or two versions in the official repos.
This is a reasonable position for most software, but definitely not all, especially when you fix a bug or add a feature in your dependent library and your Debian users (reasonably!) don't want to wait months or years for Debian to update their packages to get the benefits. This probably happens rarely for stable system software like postgres and nginx, but for less well-established usecases like running modern video games on Linux, it definitely comes up fairly often.
Of course, if possible, just saying "hey, I need these dependencies from the system" is nicer, but also not error-free. If a system suddenly uses an older or newer version of a dependency, you might also run into trouble.
In either case, you run into either an a) trust problem or b) a maintenance problem. And in that scenario I tend to prefer option b), at least I know exactly whom to blame and who is in charge of fixing it: me.
Also comes down to the language I guess. Common Lisp has a tendency to use source packages anyway.
You won't. The user may. On his system.
Many distros, and Debian in particular, apply extensive patches to upstream packages. Asking a developer to depend on every possible variation of such packages, across many distros, is a tall order. Postgres and Nginx might be able to do it, but those are established projects with large teams behind them and plenty of leverage. They might even be able to influence distro maintainers to their will, since no distro will want to miss out on carrying such popular packages.
So vendoring is in practice the only sane choice for smaller teams and projects.
Besides, distro package managers carrying libraries for all programming languages is an insane practice that is impossible to scale and maintain. It exists in this weird unspecified state that can technically be useful for end users, but is completely useless for developers. Are they supposed to develop on a specific distro for some reason? Should it carry sources or only binaries? Is the dependency resolution the same for all languages? Should language tooling support them? It's an entirely ridiculous practice that should be abandoned altogether.
Yes, it's also silly that every language has to reinvent the wheel for managing dependencies, and that it can introduce novel supply chain attack vectors, but the alternative is a far more ludicrous proposition.
That's not the idea. If a software is packaged for a distro, then the distro will have the libraries needed for that software.
If you're developing a new software and wants some new library not yet packaged, I believe you can figure how to get them on your system. The thread is about the user's system, not yours. When I want to run your code, you don't have to say:
Use flatpak; Use docker; Use 24.1.1 instead of 24.1.0; Use $THINGWhat happens is that distro developers spend their time patching the upstream so it works with the set included on the distro. This has some arguable benefits to any user that wants to rebuild their software, at the cost of random problems added by that patching that flies under the radar of the upstream developers.
Instead, the GPs proposal of vendoring the dependencies solves that problem, without breaking the compilation, and adds another set of issues that may or may not be a problem. I do argue that it's a good option to keep on one's mind to apply when necessary.
That is not what it's being asked.
As a developer, you just need to provide the code and the list of requirements. And maybe some guide about how to build and run tests. You do not want to care about where I find those dependencies (Maybe I'm running you code as PID 1).
But a lot of developers want to be maintainers as well and they want to enforce what can be installed on the user's system. (And no I don't want docker and multiple versions of nginx)
From whom? You seem to be talking only about upstream developers.
That's provided by any competent build system. If you want to build it differently, with a different set of requirements, that's up to you to figure out (and fix when it breaks).
This is also much easier for the user, since they only need to download and run a single self-contained artifact, that was previously (hopefully) tested to be working as intended.
This has its own problems, of course, but it is the equivalent of vendoring build time dependencies.
The last part of my previous comment was specifically about the practice of distros carrying build time libraries. This might've been acceptable for C/C++ that have historically lacked a dependency manager, but modern languages don't have this problem. It's a burden that distro maintainers shouldn't have to worry about.
No developer is being asked to support every distro. You just need to provide the code and the requirement list. But some developer made the latter overly restrictive. And tailor the project to support only one release process.
> This is also much easier for the user, since they only need to download and run a single self-contained artifact, that was previously (hopefully) tested to be working as intended
`apt install` is way easier than the alternative and more secure.
> It's a burden that distro maintainers shouldn't have to worry about.
There's no burden because no one does it. You have dev version for libraries because you need them to build the software that is being packaged. No one packages library that is not being used by the software available in the distro. It's a software repository, not a library repository.
You mentioned $current_debian above. Why Debian, and not Arch, Fedora, or NixOS? Supporting individual Linux distros is a deep rabbit hole, and smaller teams simply don't have the resources to do that.
> You just need to provide the code and the requirement list.
That's not true. Even offering a requirements list and installation instructions for a distro implies support for that distro. If something doesn't work properly, the developer can expect a flood of support requests.
> `apt install` is way easier than the alternative and more secure.
That's debatable. An OCI image, AppImage, or even Snap or Flatpak package is inherently more secure than a system package, and arguably easier to deploy and upgrade.
> There's no burden because no one does it.
Not true. Search Debian packages and you'll find thousands of language-specific libraries. Many other distros do the same thing. NixOS is probably the most egregious example, since it literally tries to take over every other package manager.
> You have dev version for libraries because you need them to build the software that is being packaged.
Eh, are the dev versions useful for end users or distro maintainers? If distro maintainers need to build the software that's being packaged, they can use whatever package manager is appropriate for the language stack. An end user shouldn't need to build the packages themselves, unless it's a build-from-source distro, which most aren't.
My point is that there's no reason for these dependency trees to also be tracked by distro package managers. Every modern language has their own way of managing dependencies, and distros should stay out of it. The only responsibility distro package managers should have is managing runtime dependencies for binary packages.
And if a library have a feature flags, check them before using the part that is gated.
Relying on feature flags is a pie in the sky solution, and realistically developers shouldn't have to be concerned with such environmental issues. Dependency declarations should be relied on to work 100% of the time, whether they're specified as version numbers or checksums. Since they're not reliable in practice, vendoring build and runtime dependencies is the only failproof method.
This isn't to say that larger teams shouldn't support specific distros directly, but my point is that smaller teams simply don't have the resources to do so.
Maybe my laptop is running Alpine and I patches some libraries to support musl and now some methods are NOP. As the developer, why does it matter to you?
You would want me to have some chroot or container installation for me to install a glibc based system so that you can have a consistent behavior on every computer that happens to run your code? Even the ones you do not own?
Again, this matters a lot to smaller projects and teams. Larger projects have the resources to offer extended support for various environments and deployment procedures, but smaller ones don't have this luxury. A flood of support requests can lead to exhaustion, demotivation, and burnout, especially in open source projects and those without a profitable business model. Charging for support wouldn't fix this if the team simply doesn't have the bandwidth to address each request.
Maybe go build doesn't allow this but most other language ecosystems share the same weakness.
I sometimes set up a script that runs several variations on 'cargo tree', as well as collects various stats on output binary sizes, lines of code, licenses, etc.
The output is written to a .txt file that gets checked-in. This allows me to easily observe the 'weight' of adding any new feature or dependency, and to keep an eye on the creep over time as the project evolves.
Theoretically you should be able to generate the configuration scripts through "autoconf" (or autoreconf), or generate Makefile.in for configure from Makefile.am using "automake", etc.
"go build" of arbitrary attacker controlled go code will not lead to arbitrary code execution.
If you do "git clone attacker-repo && cargo build", that executes "build.rs" which can exec any command.
If you do "git clone attacker-repo && go build", that will not execute any attacker controlled commands, and if it does it'll get a CVE.
You can see this by the following CVEs:
https://pkg.go.dev/vuln/GO-2023-2095
https://pkg.go.dev/vuln/GO-2023-1842
In cargo, "cargo build" running arbitrary code is working as intended. In go, both "go get" and "go build" running arbitrary code is considered a CVE.
It is also somewhat common for some complicated projects to require running a Makefile or similar in order to build, because of dependencies on things other than go code.
In fact, for go libraries you effectively have to otherwise `go get` wouldn't work correctly (since there's no way to easily run `go generate` for a third-party library now that we're using go modules, not gopath).
Have you actually seen this in the wild for any library you might `go get`? Can you link any examples?
Not for a library, but I have for an executable. Unfortunately, I don't remember what it was.
Runtime malicious code is a different matter. Rust has a security workgroup and their tools to address this. But it still worries me.
Of course this assumption breaks with native modules and with the sheer amount of code being pulled in indirectly ...
So many people are so drunk on the kool aid, I often wonder if I’m the weirdo for not wanting dozens of third party libraries just to build a simple HTTP client for a simple internal REST api. (No I don’t want tokio, Unicode, multipart forms, SSL, web sockets, …). At least Rust has “features”. With pip and such, avoiding the kitchen sink is not an option.
I also find anything not extensively used has bugs or missing features I need. It’s easier to fork/replace a lot of simple dependencies than hope the maintainer merges my PR on a timeline convenient for my work.
The ideal number of both dependencies and releases are zero. That is the only way to know nothing bad was added. Sadly much software seems to push for MORE, not fewer, of both. Languages and libraries keep changing their APIs , forcing cascades of unnecessary changes to everything. It's like we want supply chain attacks to hurt as much as possible.
Before this wasn't CPAN already big?
I don't deny there are some problems with package managers, but I also don't want to go back to a world where it is a huge pain to add any dependency, which leads to projects wasting effort on implementing things themselves, often in a buggy and/or inefficient way, and/or using huge libraries that try to do everything, but do nothing well.
Yes, it's a ton of overhead, and an equivalent will be needed for every language ecosystem.
The internet was great too, before it became too monetizable. So was email -- I have fond memories of cold-emailing random professors about their papers or whatever, and getting detailed responses back. Spam killed that one. Dependency chains are the latest victim of human nature. This is why we can't have nice things.
Say you need compression, you're going to review changes in the compression code? What about encryption, a networking library, what about the language you're using itself?
That means you need to be an expert on everything you run. Which means no one will be building anything non trivial.
Sure there are packages trying to solve 'the world' and as a result come with a whole lot of dependencies, but isn't that on whoever installs it to check?
My point was that git clone of the source can't be the solution, or you own all the code... And you can't. You always depend on something....
Like how xz was attacked, everyone pointed at that and no one said they didn't vet their dependencies.
That's the whole point, you attack a dependency that everyone relies on because it's been good and stable. That's how these pyramids build up over time.
So spoiler, it's not unlikely one of the dependencies in your minimal set gets exploited...
Yes, absolutely. It's the bare minimum for people offering commercial products.
Regarding the language itself, I may or may not. Generally, I pick languages that I trust. E.g. I don't trust Google, but I don't think the Go team would intentionally place malware in the core tools. Libraries, however, often are written by random strangers on the internet with a different level of trust.
Because the vast majority of development is done by people with a very narrow focus of skills on an extreme deadline, and you actually comfortable with compression, networking, encryption, IO, and all the other taken for granted libraries that wind up daisy chained together?
Because if you are, great, but at the same time, that's not the job description for like 90% of coding jobs. I don't expect my frontend guy to need to know encryption so he can review the form library he's using.
Personally, I loved it. I only looked and updating them when I was going to release a new version of my program. I could easily do a diff to see what changed. I might not have understood everything, but it wasn't too difficult to see 10-100 line code changes to get a general idea.
I thought it was better than the big black box we currently deal with. Oh, this package uses this package, and this package... what's different? No idea now, really.
I may try to put together a proof of concept, actually.
A library is by definition supposed to be somewhat generic, adaptable and configurable. That takes a lot of code.
It’s too bad that more robust languages and frameworks lost out to the import-world culture that we’re in now.
If you have any proposal how to properly manage the complexity of a FE monorepo with dozens of daily developers involved and heavy CI/CD/Devops integration, please post alternatives - given that security incident many people are looking.
It ultimately started as a small project because I got fed up with NX' antics a few years back (I think since then they improved quite a lot though), I don't need caching, I don't need their cloud, I don't need their highly opinionated approach on how to structure a monorepository; all I needed was decent change-detection to detect which project changed between the working-tree and a given commit. I've now since added support to enforce module-boundaries as it's definitely a must on a monorepo.
In case anyone wants to try it out - would certainly appreciate feedback!
I will say, I was always turned off by NX's core proposition when it launched, and more turned off by whatever they're selling as a CI/CD solution these days, but if it works for you, it works for you.
I found npm's workspace features lacking in comparison and sparsely documented. It was also hard to find advice on the internet. I got the sense nobody was using npm workspaces for anything other than beginner articles.
And npm workspaces is certainly "lacking features" compared to NX, but in terms of making `npm link` for local packages easier and running scripts across packages it does fine.
Why not print a simple counter like: ..10%..20%..30%
Or just: Uploading…
Terminal codes should be for TUI or interactive-only usage.
% echo -n "loading..."; sleep 1; echo -en "\rABORT ABORT"; sleep 1; echo -e "\rTerminated"
Admittedly, that'll spew a bunch of stuff if you're sending it to a pager, so I guess that ought to be
% if [ -t 1 ]; then echo -n "loading..."; sleep 1; echo -en "\rABORT ABORT"; sleep 1; echo -e "\rTerminated"; fi
They are supposed to do things like ul utility does, but neither BSD more nor less handle when a CR is emitted to overstrike the line from the beginning. They only handle overstriking characters with BS.
most handles overstriking with CR, though. Your output appears as intended when you page it with most.
Now the threat is: when they “improve” it, you get that automatically.
left-pad should have been a major wake up call. Instead, the lesson people took away from it seems to have mostly been, “haha, look at those idiots pulling in an entire dependency for ten lines of code. I, on the other hand, am intelligent and thoughtful because I pull in dependencies for a hundred lines of code.”
Maybe scolding and mocking people isn't a very effective security posture after all.
I'm not trying to be effective, I'm just lamenting. Maybe being sarcastic isn't a very effective way to get people to be effective?
I'd say it all depends -- there's that word again -- on what those 100 LOC are expressing. I suppose one could still copy/paste such a small amount of code, but I'd rather just check in some subset of vendored dependencies. Or maybe just pin the dependency to a commit hash (since we can't depend on version tags being immutable). Something actionable beyond peer pressure at any rate.
The big advantage with a dependency manager is that you don't have to find all of the dependency's dependencies, figure out the right build settings, etc. That's super helpful when it's huge, but it's not really doing anything for you when it's small.
Compared to typescript where it’s a package + code to use said package which always was more loc than anything comparative I have done in golang.
So your suggested approach does not seem to scale well.
At some level of complexity it probably makes sense to import (and pin to a specific version by hash) a dependency, but at least in the JavaScript ecosystem, that level seems to be "one expression of three tokens" (https://www.npmjs.com/package/is-even).
I've been preaching this since ~2014 and had little luck getting people on board unless I have full control over a particular team (which is rare). The need to avoid "reinventing the wheel" seems so strong to so many.
it's common that the part that I actually need is like 100 LOC rather than 1500 LOC.
Please keep preaching.
This week, I needed to add a progress bar with 8 stats counters to my Go project. I looked at the libraries, and they all had 3000+ lines of code. I asked LLM to write me a simple progress report tracking UI, and it was less than 150 lines. It works as expected, no dependencies needed. It's extremely simple, and everyone can understand the code. It just clears the terminal output and redraws it every second. It is also thread-safe. Took me 25 minutes to integrate it and review the code.
If you don't need a complex stats counter, a simple progress bar is like 30 lines of code as well.
This is a way to go for me now when considering another dependency. We don't have the resources to audit every package update.