This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level. It's an issue in Debian because they can't handle multiple concurrent versions (beyond renaming the package) and want every package built without internet access.

But that same issue affects Ruby, Python, etc. if the project specifies strict versions. And if you deal with filesystems, you really want strict versions - I understand why the author wants to keep it that way.

It's more of a self-inflicted Debian policy pain rather than anything to do with Rust. The author could be nice and vendor, but also he's completely right to refuse. The request is basically "we put handcuffs on ourselves, so please you do the work instead".

In fact, I am tired of the same issue e.g. when evaluating machine learning tools with Python. It's as if everyone just says "dependency = ${exact_version_I_have}" and never considers anything else. Not helped by the fact that most of these projects break ABI every other day which I still think it is just plain evil.

Please don't say "just use containers/virtualenvs/vendoring/whatever" because at some point you obviously want these "containers" to use your real hardware. And your GPU driver is only going to be at one version, no matter how many statically linked executables you have.

> And your GPU driver is only going to be at one version, no matter how many statically linked executables you have.

This is a non-issue.

libcuda.so (which is part of the NVIDIA driver, their userland part) can not be statically linked into anything, since it's closed-source and no .a shipped at all, so no possibility of misconfigure it to be statically linked. And NVIDIA drivers, as a whole (including a pair of libcuda.so and corresponding .ko), is infinitely backward-compatible, you can always use new driver with old libraries. So the problem you mentioned does not exist at all.

Statically linking to cuBLAS / cuDNN etc never add more pain and can only reduce pain. Unless you hold it wrong.

That is hardly true for NVIDIA over the years, as I have experienced personally, and it's definitely not true for about every other GPU driver in existence -- particularly AMD's RoCM, which even recommends specific Python versions to go with each ROCM version. In addition, replace "GPU" with any other piece of hardware, such as NPU. Or even software, such as your desktop environment. Do you think a statically linked binary will run with Wayland+1 ? It's already a chore to run games that link with older SDL versions, and the only reason you can at all is that SDL was dynamically linked...

Even the kernel itself does a pretty poor job at maintaining backwards compatibility (e.g. contents of /sys changing every other year). Statically linking just hardcodes one of these things, and then forgets about the rest, such as IPC APIs, desktop APIs, filesystem structure, etc. which due to the poor culture out there are moving as much as the libraries themselves.

> Do you think a statically linked binary will run with Wayland+1 ?

Yes, yes it will. The baseline stuff (the base protocol and xdg_shell) has been stable for years. Of course, one might argue that the caveman byte poking provided there turns a modern machine into a slightly faster 15-year-old one, and I agree it does. (It’s still fun to do as an experiment.)

But the moment you try to actually leverage the GPU the modern Linux desktop forces you to abandon static linking in order to load pieces of the graphics driver into your address space, and unfortunately those pieces will in turn force you to dynamically link the distro’s preferred libc (likely Glibc) and libwayland (as opposed to any other protocol implementation).

So either you poke pixels into a byte buffer by hand, and your question gets an answer in the affirmative simply because that foundational layer is fairly small, or you take advantage of the hardware, and your question becomes moot because you can no longer statically link.

In Rust, it is frowned upon to pin to an exact version. We do encourage people to specify what semver version range will work. We don't have great support for depending on and verifying multiple-major version ranges when a library broke compatibility but not in a way that affects you.

Is this being tracked somewhere?

It's 100% culture: this happens across all languages.

> It's as if everyone just says "dependency = ${exact_version_I_have}" and never considers anything else.

This is what I call "version soup".

The idea that every project can choose an near-arbitrary set of very exact version numbers of dependencies and expect it to work. And that every project on earth has the burden of continually stirring their soup, often through a tool like dependabot, hoping (cross fingers) no compatibility problems come about.

Of course, there's no guarantee that those versions will work together. The extremely limited abilities packaging tools have for expressing dependency restrictions (>= this or that version) generally lack the ability to even handle the concept of stable branches with backports. i.e. "No, it must be >= 1.2.3.. what do you mean they backported the relevant fix to 1.1.4? 1.1.4 < 1.2.3 so it's unacceptable". The way these specifications are then used by authors brings an extra layer of noise to the compatibility situation - very few (understandably) will actually go and check the limits of version compatibility.

In nixpkgs we attempt to address this situation for the python ecosystem by providing one version of each package (with few exceptions) per release. But in return, we put in work to make sure those versions actually work with each other - generally by getting the projects' test suites integrated into the build system. The idea is that an app built to depend on nixpkgs packages should be able to expect to do dependency upgrades as a "jump" every 6 months when there's a new nixpkgs release, but otherwise be able to depend on a stable suite of packages that still receives security updates & backports.

> In nixpkgs we attempt to address this situation for the python ecosystem by providing one version of each package (with few exceptions) per release. But in return, we put in work to make sure those versions actually work with each other - generally by getting the projects' test suites integrated into the build system. The idea is that an app built to depend on nixpkgs packages should be able to expect to do dependency upgrades as a "jump" every 6 months when there's a new nixpkgs release, but otherwise be able to depend on a stable suite of packages that still receives security updates & backports.

Is this different from any other Linux distribution?

Essentially no, but

- most Linux distributions don't reach very far into e.g. the python ecosystem (python packages that exist are generally there to support packaged applications)

- nixpkgs isn't linux specific - most packages work on macos, and nixpkgs happily works on top of other distributions.

It should be noted that, particularly for rust, it's a silly policy as rust does not support dynamic linking of rust libraries. All rust applications are statically linked.

So trying to force all projects using foo onto the same version of foo is just a huge headache with no real benefit to anything.

> So trying to force all projects using foo onto the same version of foo is just a huge headache with no real benefit to anything.

It creates a security nightmare.

Distributions expect to be able to security patch without waiting for laggard upstreams to bump their pinned dependency versions. If the ecosystem insists that only upstream-sanctioned versions are acceptable, then that runs contrary to this expectation, because it blocks distributions from patching or bumping dependency versions. If this case should be an exception, then you end up with a security team having to patch myriad different versions. That's the security nightmare.

Fair point.

I do think we need to reevaluate how security is managed in terms of software. It's often the case that we say "this library version has a CVE, we need to update it!" However, it's not often the case that the CVE in the library is even exploitable from the parent application.

I'm thinking of a recent 9 CVE against zlib... But actually it was a rarely used extension API for working with compressed files.

Something like path tracing to see if a vulnerable API is even accessible from the parent application/library before instructing the universe to adopt a new version.

Rust has the advantage of aggressively gating many library APIs behind feature flags as a convention. A lot of third-party libraries consist of a core of the most essential functionality that the entire library just can't go without, and everything else, particularly things that require yet more external dependencies to work, is gated behind a feature.

You could imagine a model where security vulnerability reports include a combination of features required to trigger that vulnerability. If the vulnerability was in a rarely-used extension, like it was for Zlib, many dependents would be automatically marked as non-vulnerable because they wouldn't have the necessary feature flag active.

I think it's already complicated to find all the software that uses a library version that has a known vulnerability, because of the many slightly different versions floating around, static linking, software running in docker containers etc. Expecting to further identify which applications use a particular part of a library gets even more difficult, and lazy developers will use this as an excuse not to patch far more often than you'll save on unnecessary updates.

Not to mention, this already happens to some extent, especially for older versions of software: distributors and developers do sometimes say "we will not provide a patch because even if we use the affected library, we don't use it in a way that triggers the vulnerability".

Not all distributions suffer from this, because they have the tooling to patch all crate versions given a name. Compile times are actually the worst part but can be mitigated. But these distros also fundamentally accept the language's compilation model in question, so they have to develop solutions if they want to supply software to their users rather than make up excuses. Most distros don't like vendoring, for reasons. But many code maintainers do like it, for reasons. So it is what it is.

There are also more tools than ever to do things like correlate versions of OSS packages with reported vulnerabilities e.g. Google's osv.dev. 95% or more of Rust vendoring isn't actually "vendoring" in the sense you have to figure anything out. The dependencies are literally written in the lock file with the version and that can be traced back to a Git hash automatically in almost all cases. Some crates that bind C libraries might require this work, where you have to look at the code to see "What version of the code did the `cp -R` into here?" but they are practically limited in number, and detectable. You aren't doing any detective work for 98% of this stuff. It can be scaled pretty well with a few (good) heuristics.

Rust even has the advantage that large, vast amounts of code is actually statically analyzable under known sets of configurations/features. In the future I suspect a tool like `govulncheck` for Rust will be developed, making accurate vuln tracing even easier.

While I'm admittedly biased because I work on a distro where this problem isn't as directly relevant, my answer is that in 2024 if you don't have the tooling to do this stuff, it's absolutely on you. Whether as a policy decision or a technical limitation, whatever.

What do you mean that they can patch all crate versions given a name? Sometimes the same patch might be trivially applied to many different versions of a library, but in the general case, this is not true: the amount of work is in direct proportion with the number of library versions you have to patch, even if you know exactly which versions those are. So the difference between maintaining, say, 3 versions of a library, and 30, is going to be ~10× the work, even with tools that instantly identify the exact versions that need to be patched.

Tooling to apply the patches is trivial. This isn't the problem I'm describing.

As the sibling comment says, the issue is when the patch doesn't apply cleanly to some subset of versions in use, or even worse if they appear to apply cleanly but leave behind logic bugs.

If distributions did approach this by "distro-patching" specific versions, then that would reintroduce the upstream complaint that distros are using versions of dependencies not sanctioned by them. So your approach, even if it did work in the general case, would take you back to square one.

Yeah, I don't think either of these are really big problems. People identify the vuln, the fix, and they tend to share the patches for backports across versions + across distros, and/or immediately update downstream users to fixed versions because, again, the nifty lockfiles literally tell you if they are vulnerable as they contain the dep graph. You query every package's dep graph and look at the bounds. Maintainers with RUSTSEC advisories will yank the crate on crates.io, forcing upgrades across the stack. You can apply patches to most versions of a crate (again: you can find all of them!) and loosen or make more targeted fixes if needed. But even if 30 versions are vulnerable, in practice 30 patches don't need to be written in the p99 case. So I just disagree with the sibling on that, based on my experience. Once you've verified a targeted fix cranking out patches is the easy part. Most of these packages aren't the velocity of the Linux kernel. They aren't rewriting 10,000 lines of code a version so you have to re-interrogate the fix.

But most importantly, I don't see any argument this is all meaningfully more work than all the manual shit Debian maintainers do to get all this stuff working, on top of the fact they try to ship frankenstein versions of packages that silently break and piss off users (every distro ships some busted packages but it's a matter of give and take.) Other maintainers of other distros don't spend time on all the stuff the OP spent time on. They don't spend time loosening lockfile bounds on Rust crates for filesystem-sensitive tooling(!!!) all in an effort to make a 35-year-old cultural rule about C/C++'s compilation model apply to them. Mostly, they accept life, run some script like 'import crates.io package XYZ version 1.2.3', commit the result, and get it code reviewed and merged. They can then spend time on other things. And often, these same processes are used for all updates, including many security updates. This stuff is insanely well oiled in other projects.

Sorry to be the hater, but this is squarely a Debian dysfunction as far as I can see? If they spent more time actually solving infrastructural problems they had, they would be able to spend time on actually delivering more working software to their users in the world they actually exist in.

> in an effort to make a 35-year-old cultural rule about C/C++'s compilation model apply to them

> this is squarely a Debian dysfunction as far as I can see

Nothing I have said is specific to Debian or to C/C++. I've presented general arguments that hold regardless of distribution policies or the language toolchain used. Nothing you have said demonstrates why the general points I have made would apply to Debian or to C/C++ but not to Rust-based projects.

If you still believe that this is the case, then I think that demonstrates that you do not understand the matter deeply enough.

The only thing that Debian is insisting on is basic software supply chain hygiene. Every so often, there is a very public failure in ecosystems that don't bother with this hygiene, while Debian continues unaffected.

> You can apply patches to most versions of a crate

The point of the complaint that spawned this discussion is that there's an upstream that considers it unacceptable for downstreams to be doing this patching (or indeed running any dependency version that is not sanctioned by upstream). It is a contradictory position to justify downstream patches as a supposedly easy solution while at the same time complaining that downstreams patch at all.

The fact is that distributions users do not expect to be beholden to laggard upstreams while awaiting security fixes. This requires distributions to modify the dependency versions (or patch, which is the equivalent "off-piste" behaviour) used to build their packages.

If the only way a security team can cope is with a single version policy, that sounds to me like a tooling problem.

Take Nixpkgs, for example. They have this concept called "default crate overrides" which their rust builder uses, which offers the capability to patch all versions of a crate. That's not to say that Nixpkgs does rust perfectly, but just an example of the sort of tooling a security team should have at their disposal.

The security team got used to c/c++ being the only way to build native binaries, and would prefer to peg other square languages into this round hole.

FUD enuses.

How does it patch all versions automatically? If the library has been refactored, you still need to modify the patch for the pre-refactor and the post-refractor versions, for example. The closest you might get is a ChatGPT-based tool, but that is nowhere near safe enough for security patches.

I didn't say it's automatic. You can have overrides that target a more narrow range of versions, if you need. If there has been a major refactor, then you need two patches. It's not rocket science, and you definitely don't need chatgpt.

You can also blend both strategies - you have have a version minimization strategy, where you only use, for example, one major version of a library at a time.

> rust does not support dynamic linking of rust libraries. All rust applications are statically linked.

That's not true.

  $ ldd /usr/bin/rustc
   linux-vdso.so.1 (0x00007fde30c44000)
   librustc_driver-adb68da129cf8bba.so => /lib64/librustc_driver-adb68da129cf8bba.so (0x00007fde2cc00000)
   libstd-aa46ce58aa106bb2.so => /lib64/libstd-aa46ce58aa106bb2.so (0x00007fde2cad3000)
   libc.so.6 => /lib64/libc.so.6 (0x00007fde2c8e2000)
   [...]

The Rust compiler itself (which is a Rust application) is dynamically linked to the Rust standard library and to a compiler driver library.

What Rust currently doesn't support (there's a desire to fix this someday), is dynamically linking with a Rust library compiled with a different version of the Rust compiler, unless the library has been restricted to a C-compatible ABI. That's because Rust currently does not have a stable ABI; things like structure layouts can change between versions of the compiler, and Rust makes heavy use of inlining.

> What Rust currently doesn't support (there's a desire to fix this someday), is dynamically linking with a Rust library compiled with a different version of the Rust compiler

Note that even with the same compiler there a lot of factors that must remain the same to make the dynamic linking work properly. For example changing any compiler flag or dependency feature flag will likely result in incompatibility. The best way to guarantee that the dynamic library and the executable that loads it are compatible is to build everything as part of a single workspace (and yes, this means that if anything changes in the dynamic library crate then the executable that depend on it must also be rebuilt).

> It should be noted that, particularly for rust, it's a silly policy as rust does not support dynamic linking of rust libraries.

Almost. You can write dynamic libraries in Rust (using the cdylib target type), but their external interface will be limited to the C ABI. For all intents and purposes, they'll be 'C shared libraries' that just happen to be written in Rust. (You can also create C headers for the exported C API using cbindgen.)

Almost—even if you need to rebuild dependencies, you can still patch security issues just once instead of doing so in every vendored copy. That's the first point in the parent comment.

The same applies to Go.

It remains a problem for source packages, which are an important part of Debian necessary for offline and reproducible builds.

> This isn't Rust specific. The same issue exists in all languages where the versions can be restricted at project level

Can confirm, I occasionally use an ai program that has to hard code all of their python dependencies, because it's the only way to get it to compile, let alone run properly... and then they go and change the underlying package manager, and figure out they have to hard code even more... it's a bloody mess, but thankfully you can just run it as a docker container once it's all working...

I think that 2 different things are mixed up.

For Rust it is language specific, because the thing being statically built, doing differently is very hard.

For other languages like Python, there is not a need and you should not do that. The dynamic part of it make it so that developer should be resilient to different versions. And python also provide a lot of conveniences for that (just think about six).

But in your case, you faced the current situation of lot of "data scientist" jumping the band wagon of AI are creating a lot of things like apprentice sorcerers without a real experience of software engineering.

> For other languages like Python, there is not a need and you should not do that.

Normally I would agree, but when it comes to cuda and rocm, and the one that intel uses... you find that you have to pin torch/audio/video, and then you have to start pinning its dependencies (and their dependencies etc etc) or else you start getting random build or runtime failures...

No, in both cases the issue is the same: whether the actual interface of the library is staying the same or not, and over what time period. Python packages have plenty of the same kind of breakages. usually it's unintentional, a bug or similar, sometimes it's a deliberate breaking change in a point release. Either way the effect is the same. Version X works in project A but newer version Y doesn't, and the reverse can be true for project B. It's just that in python you're most likely to get the problem when you run the code, wheras compiled languages can also fail at the build step as well as at runtime.

It can be pretty bad with the bigger ML/scientific computing packages, hence all the version pinning that happens, though I will concur that generally speaking AI researchers don't understand package management in the slightest and you get all kinds of horrors as a result (e.g. mixing anaconda and pip in the same project, bonus points if it's blasted over the distro's python packaging for even more chaos. works on one very broken machine if you are lucky and don't touch anything).

Debian's approach is similarly frustrating: they will try to make your code work with an older version of a library than the one you developed with, which doesn't even work within semver (semver only defines backwards compatibility, not forwards). That's what caused the problem in this case: rolling forward dependencies was not recommended but didn't break anything, rolling a dependency backwards meant it stopped working, which is something absolutely no-one should be surprised or upset by (at least it meant the build broke as opposed to debian shipping a broken package like would have likely happened with python. It just meant that debian was shipping a version with bugs fixed in upstream). If you're going to do that you've got to be prepared to take responsibility for fixing the problem that you've introduced.

Not disagreeing, but IME much more often I have to run something that locks specific versions of their python dependencies because ... well ... why not -- kool kids run everything in a virtual environment anyway.

Version locking is a powerful tool that should be used very sparingly. My 2c.

  if you deal with filesystems, you really want strict versions

How are doing everybody else ? What is specific with filesystems, here ?

This is a bad dependency management, nothing more. Vendoring is a PITA.

Level of potentially messing up. If you have a chat app, relax the deps and it crashes - oh well. But if you have a tool which can cause data loss, you really want to be sure everyone's running what you tested.

It's similar to why restic (backup software) has official, static, reproducible binaries in their releases.

And if my kernel crashes ?

And if whatever library bugged, oh well, random people can access my systems ?

And if my bgpd bugged, oh well, we just inject random routes et forward data randomly ?

Makes no sense.

Your kernel has no external library dependencies.

Relaxing library requirements can introduce security issues when the dependency behaviour changes, so I'm not sure what the argument is.

If your bgp is bugged, yeah it sucks, but it's recoverable. We're using encrypted traffic these days to ensure that even captured traffic is safe.

The difference here is that data corruption is not recoverable.

Linux uses at least libc

Linux kernel doesn't use libc. Libc provides the interface for userspace to call the kernel.

It does includes its headers (stdlib.h, at least), in various subsystems.

In user space tools and tests only. The kernel part does not include stdlib.h anywhere. Nowhere where it matters in the context of "And if my kernel crashes?"

You are right, thank you for the insight

Oh well, it was probably just a crash in my Internet connected software where users can just sent inputs to be processed on my machine.

RCE would have been worse than file system bugs, as it leads directly to a superset of problems including potentially losing all your files.

Filesystems are complicated and have nasty failure modes. Strict versioning reduces the impact of software development QoI properties, i.e. that basically all code is substantially wrong.

I would expect it to be the opposite (especially for a linux file system): you want to test with as wide a variety of versions as possible, because that what you're going to encounter in the real world. If your code only works in a limited set of cases, what happens when you mount the file system on a different system, is the file system going to be trashed?

These are all compile time dependencies in this case. You're not going to encounter a variety of versions - that's exactly what the strict version bounds are doing. The app will be built with the tested version and everyone gets the same results.

There's at least 3 different versions that can happen (the filesystem version, the kernel version (including stable backports), and the tooling version). I'm presuming udev being a dependency means bcachefs links to that (which will be different on different OS versions).

Those are completely different things than the dependencies you can make a choice about. You have to handle different filesystem/kernel versions regardless - you don't get a choice here.

The author would be fine with vendoring. The problem is that Debian doesn't allow it.

> The problem is that Debian doesn't allow it.

Vendoring used to be more common, even in Debian. But then there was an important zlib vulnerability (which IIRC could be exploited through manipulated compressed data), and they had to chase and fix all the many copies of zlib embedded all over the place. To make things worse, some of them were not only old versions of zlib, but also had modified that zlib code, so each had to be reviewed before applying the fix.

Debian (and other traditional distributions) learned from that incident, and started a strong push to not only remove vendored copies of libraries, but also when possible use a single version of these libraries. Which means that, whenever an important security issue is once again found in a widely used library, only a single copy has to be fixed and updated.

I'm interested in hearing more about the history behind Debian and zlib! I did some searching and the closest thing I could find was a nod to the same incident in the Upstream Guide[0]. Do you know of a place where I could read more about it?

[0]: https://wiki.debian.org/UpstreamGuide#No_inclusion_of_third_...

It's been so long ago, that it's hard to find all the discussions I had seen back then. I recall that it was after a long time without any zlib release, so looking at the zlib history, I think it was this one fixed in zlib 1.1.4 from 11 March 2002: http://www.zlib.org/advisory-2002-03-11.txt

Looking at the debian-devel archives around that date, I found a Debian developer complaining about the vendored zlib copies (https://lists.debian.org/debian-devel/2002/03/msg00716.html), but not the full discussion about getting rid of vendored libraries, so it must have happened elsewhere.

Thanks for taking the time these references up for me!

And that's because the users of Debian are not fine with vendoring.

it is rust specific because rust is the first attemp to replace proper system engineering languages with one that while nicer on memory management and overall ergonomics, is worse in tooking. cargo brings many malpractices from java/JavaScript (maven, npm, etc) that were always shunned in systems engineering and, mark my words, will be a security nightmare for linux in the near future.

What makes a system engineering language "proper"? And what languages besides C fulfill those criteria?

The grammar might be confusing, but the comment you're replying to clearly considers rust a (new) proper systems language, and nicer in many ways (memory management and overall ergonomics) than C, but with worse (from the perspective of systems programming) tooling.

thanks for making it more succinct. exactly.

Go, D, Ocaml, Nim have the same issue, most are older than Rust and are arguably systems languages.

C/C++ built with Bazel or Conan will have the same issue too.

those are userland system applications at most... which i grant is also what the original article talked about, but still.

Go would only be comparable if you had hundreds of vendored dependencies but Go's community and culture have encouraged use of the standard library over external dependencies where possible. A core problem with Rust is the lack of an adequate standard library. The problem with Cargo is when you have an application with hundreds of dependencies. That does not exist in any mainstream Go application.

1. Why Rust Libraries May Never Exist - https://www.youtube.com/watch?v=769VqNup21Q

2. The Rust Cargo ecosystem is broken - https://www.youtube.com/watch?v=OdMQwDtumJ8

Having a big standard library doesn't help that much since it needs separate updates. Every other golang app I use requires updated crypto, net, or sync. In theory they come in the standard library, but the moment you need updates, they're exactly the same as other dependencies for packaging purposes. (it's very common, yes I do packaging)

Go is unlike other languages mentioned here (and this arguably applies to OCaml too).

As much as its standard library superficially looks close to C, it is as equally unlike C in its performance and abstraction level. It does not have suitable tools to do systems programming properly and its compiler and interop capabilities are too weak.

I think this is the same shape of issue that I’ve experienced with Debian for as long as I’ve used it - close to 15 years now.

Debian is a great OS, but it targets stability and long term support for its releases. That just isn’t compatible with newer, faster moving software that’s still working towards stability. I remember it being an issue when I was playing around with Mono around 2010 ish, and it’s an issue now with bcachefs - a very new and fast moving technology.

For motivated users, the solution is to install bcachefs-tools direct from upstream, or from a third party packager (if one exists). When bcachefs stabilises, I’m sure it’ll find its way back into Debian official.

What I think is somewhat interesting in this case (if I've read the post correctly) half the changes were for newer versions of dependencies. If that's the case, then I'm inclined to start wondering what's going on with bcachefs upstream (and the whole dependency tree), if they're not keeping up with versions.

Imagine that an update of bcache-fs is released and 3 dependencies (could even be transitive) were updated and the newer versions were not packaged in Debian. The maintainer now has one package to update but another 3 to package for the first time. That's additional work. AFAIK Fedora has a tool to automatize this work, but maybe Debian has additional requirements that make this harder.

Debian (and derivs) have similar tooling (and the vast majority of rust packages are tracked in a monorepo https://salsa.debian.org/rust-team/debcargo-conf).

My point was "some of the upstream versions were older than Debian's" which means the "Debian is behind, Debian is slow" is not true in the case. That usually only happens where an upstream is either very conservative with bumping versions, or where upstream is under-maintained, neither which at first glance seem to be true of becache-fs.

My read is that whatever subset of the ecosystem bcachefs-tools just hasn’t stabilized. Presumably it will, but once everything is functionally at 1.x even if they don’t reach it officially this will be become a non-issue.

That said, having dabled in rust there’s a temptation to name breaking changes to improve ergonomics because the type system is so expressive. I almost wonder if this is an argument for simpler type systems to reduce churn.

It looks like the two packages that were bumped forward are ones where the changes were very minimal (it's not obvious why errno even bumped their 'major' version, and udev seem to do it on every release even when backwards compatible). Also errno 0.4 doesn't seem to exist, so I think that's a typo.

(There is a somewhat frustrating trend in rust packages for them to sit on 0.x versions for a very long time, even if they're fairly heavily used in the ecosystem and pretty stable in practice)

From what the author of bcachefs is saying, the forward moves were not a problem, it was moving bindgen backwards which broke the build, meaning the packaged version of bcachefs-tools stayed very out of date, causing him to get bug reports for known and fixed fairly serious bugs (ones which rendered machines unbootable).

I love Debian for this stability. It is a great host OS for all my VMs.

I agree. Debian is a waterfall OS living in an agile world. There's a fundamental mismatch between Debian's philosophy and the reality of today's open source software ecosystem.

Maintenance is much more practical if everything in your dependency tree uses the same version of every dependency. Achieving this in practice is much of the work distribution maintainers have to do.

Doing this has a couple of key advantages:

1. when a security vulnerability needs patching in a stable distribution release, the security team only have to patch one version instead of a dozen or more different versions bundled all over the place

2. a library that depends on dependencies A and B, both of which depend on X, can actually work properly if it needs to pass API objects created by X across between A and B, since X is of the same version

In an ecosystem where it's considered acceptable to "pin" versions of dependencies and also call any system that doesn't use the pinned versions "unsupported", both of the above two cases become impractical.

Whether you use shared libraries or static libraries, the above matters still exist.

> 2. a library that depends on dependencies A and B, both of which depend on X, can actually work properly if it needs to pass API objects created by X across between A and B, since X is of the same version

That feels like an advantage for the developer, not the distro maintainer.

I find that amusing, because you're saying it's easy if they can use one version of the library. Well, that's also true if they're writing an application, hence pinning.

> That feels like an advantage for the developer, not the distro maintainer.

It's an advantage for the ecosystem, yes. Not the developer of the library (X in my example), but a different developer. Someone who is trying to write an app to do something interesting with A and B.

> ...because you're saying it's easy if they can use one version of the library. Well, that's also true if they're writing an application

I don't think that's true.

Perhaps library A has pinned dependencies for X such that it is in conflict with unrelated library B that also uses pinned dependencies for X. Both libraries are now less useful to the ecosystem, because they are now inadvertently incompatible with each other (if their APIs expose X's objects).

My example demonstrates why this doesn't work in the general case, and therefore why it's wrong to expect that everyone does it this way.

No.

1. Rust is as hostile to dynamic libraries as glibc to static.

2. With everything static you have to rebuild every dependent on any security patch anyways. If you meant with multiple versions maintainers have to backport patches to multiple versions. Maybe don't backport at all? Some people appreciate having backports. Users of software written in Rust does NOT. So why bother doing backports for them?

3.

> a library that depends on dependencies A and B, both of which depend on X, can actually work properly

This will be detected during compile (since it's all static! sorry dynamic linking fans) and dealt with.

Dynamic libraries have had interface versions and linking strategies for decades.

For statically compiled apps, if the version is over specified recompiling will have no effect. The dependent will have to be updated with a new version string.

> This will be detected during compile (since it's all static! sorry dynamic linking fans) and dealt with.

If it is correctly detected, then the only "dealing with" that can be done is to fail to build.

> Some people appreciate having backports. Users of software written in Rust does NOT. So why bother doing backports for them?

This is completely false. First and foremost, people who use applications don't care at all what language those apps are written in, and there is no "community of people who use Rust tools" that could have a different world-view from everyone else.

And beyond that, there is no one in the world who doesn't want backports, because it's never safe to just take the latest version of some piece of software and expect that everything will still work as expected. Having to take even a month of new features just because you need a security fix.

And note that Rust even at the language level has strong support for backwards compatibility and support for older versions. It's definitely not true that Rust people, even developers, live at the bleeding edge and don't need backports.

Maintenance is much more practical when you use the versions upstream tests in their CI and not whatever mishmash of ancient/silently incompatible deps that each distro separately decides to combine together.

Upstream CI isn't magic that only they have. Distributions run the same tests upstreams do if they are made available.

If tests falsely pass, then that's a quality problem in the upstream tests.

Generally speaking libraries don't pin versions, applications do (in rust-land, a Cargo.lock is respected for the folder you're building, not your dependencies. A dependency can specify an exact version if it wants but an application can override that kind of thing, and it's generally not considered a good idea). This makes 2) a non-issue for the ecosystem (if an application needs to pass objects from X between A and B, then they'll need to pin to a single version of X). 1) is more of a disadvantage, but it's unclear to me that the effort distro maintainers put in to fix to a single version actually results in a reduction in effort overall.

One package's app is another package's dependency, so this distinction cannot work in the general case.

It can work perfectly well, at least as far as API/ABI compatibility. An app is an executable, a library is a static or dynamic library. Something isn't generally both at once (a project or package might have a library and some tool applications related to the library, but those tend to get split out by distribution package maintainers anyway, and with cargo there's always a top-level package which sets the pinning, the rest are libraries whether or not they're the top level in other contexts). If you're talking about IPC or RPC, then yes, this matters more in terms of compatibility, but that's an area where a lot more attention is paid to compatibility in the first place, by everyone.

As others have said this isn't just Rust, distro "release" packaging is untenable now for non-base-system packages: there are an ever increasing number of packages, they release with updates users need/want at a rate far faster than the distro release schedule and have too many conflicting dependencies.

To deal with this today you install a stable base system with an overlaid "user" package system so Debian/Fedora + nixpkgs/guix/brew/snap/flatpak/mpr/pip/cargo/etc. Unfortunately because because although you can get most packages on nixpkgs you can't get all and you'll need to install multiple so it becomes a nightmare to maintain (how do you remember all the package managers you have to update during the xz vulnerability?) and extremely bloated due to duplicated dependencies (especially for GUI packages that require gnome or kde).

You do get pretty far for CLI-only packages by just adding nixpkgs though. Too bad it's so terrible to use.

MPR is also pretty interesting: You basically leverage the AUR for Debian and with a few changes you could probably make the dependency names translation automatic and transparent. It solves the bloat problem (since you'll mostly use system packages) but doesn't help with library versions.

Shouldn't the filesystem utilities be base-system packages, though? Like e2fsprogs, xfsprogs ... I guess rust just isn't appropriate for base-system packages, because the dependencies move too fast, and so rust components need to depend on different specific versions of all dependencies, and are "horrifically out-of-date" in just a few months ... and rust developers wouldn't stoop so low as to recognize that e.g. bindgen is a particularly hairy and touchy dependency so they should just vendor the bindings output by bindgen, rather than the entire bindgen tool plus all other dependencies in full ...

> and rust developers wouldn't stoop so low as to recognize that e.g. bindgen is a particularly hairy and touchy dependency so they should just vendor the bindings output by bindgen

Why can't Debian do that though?

Moreover vendoring is generally considered a bad practice, more so by Debian, so this just seems a step backwards.

I'm not that familiar with bcachefs-tools and was speaking more generally but I thought bcachefs-tools wasn't critical for normal booting (On Ubuntu 24.04 nothing seems to depend on that package and it seems optional. The same for btrfs-tools). Some other comments suggested you need it to mount a degraded file system but it seems to me that is a case where either 1. your system can't boot and you need external recovery boot tools anyway or 2. you can boot and can run whatever tools you want (you can install the latest version of bcachefs-tools via nixpkgs for example). The real problem there seems to be that the program was silently broken due to the changes made to get it to build on Debian.

Though I agree that it is closer to a system-critical package than most packages.

IIRC the brokenness was simply due to a bug in an upstream release of bindgen, which "was quickly fixed upstream, but as far as I know it's still broken in Debian" (which must have some reason of not updating bindgen again ...) so it wasn't some subtle incompatibility or modification to bcachefs-tools IMHO, it was just bindgen.

Gentoo has SLOTs and they work well. But for Go and Rust, I think they chose the vendoring path because it's just too damn Don Quixotesque to try to mirror their NPM tier package repositories.

> Developer on reddit clarifies that it builds and does the wrong thing, and recommends dropping Debian as a solutio

Can you link to the post on Reddit you are referring to?

I think this is the comment:

https://www.reddit.com/r/bcachefs/comments/1f4erbg/comment/l...

> rust insists on a very specific version of various dependencies

It only insists on semver-compatible versions (if a Rust/Cargo package specifies libfoo=5.1, it will work with libfoo=5.9). It's one per major version, not that different from Debian packaging "libfoo5" and "libfoo7" when both are needed.

The difference is that Cargo unifies versions by updating them to the newest compatible, and Debian unifies them by downgrading them to old unsupported versions, ignores compatibility, and reintroduces bugs, and disables required functionality.