If you have a thousand packages linking statically against zlib, you will have to update a thousand packages in case of a vulnerability.
With a shared zlib, you will have to update only one package.
I think that having a sophisticated desktop OS where every process had distinct physical memory for duplicated uses of the same code would be problematic at scale, especially on systems with less RAM. At least that physical memory would still be disk-backed, but that only goes so far.
That said, the security argument is also a good argument!
- KSM only works on pages that are identical; I am skeptical that it can actually identify 2 instances of a lib after it's been through an optimizing compiler+linker (mostly LTO)
- KSM has performance overhead
- zram/zswap only let you reduce the impact of running out of memory at the cost of performance; you're still swapping, no matter how much we improve the details, so it'll always be worse than outright deduplicating libraries
I disagree that swap is only (or even primarily) for situations where you're running out of memory, though. It is useful for reducing I/O contention and improving memory utilization by replacing underutilized pages with file cache when it makes sense to do so. See this for more details: https://chrisdown.name/2018/01/02/in-defence-of-swap.html
And yes I sort of agree with swapping unused stuff so you have more room for cache, but that's only good for unused stuff; if you swap the programs you're actually running - even just zram swap - you're going to hurt performance.
Zram/zswap isnt nearly as good as the kernel knowing straight from the outset that all processes want exactly the same mappings from the same files.
And on a distribution like OpenSUSE TW, the automation is set up such that those thousand packages do get rebuilt anyway, even though they're dynamically linked.
(*): ... and build time on the distro package builders, of course.
This recently (ish) bit hyprland, because they used a pinned version of wlroots, and the version of wlroots on Debian wouldn't work. And they couldn't just dynamically link to this pinned wlroots because that's still not around.
If the version of wlroots in Debian, hypothetically, contains a vulnerability, then only one change is needed instead of two.
I would argue if you allow static linking you implicitly encourage vendoring. I can't imagine such a distro where static linking is allowed, but there are 0 packages which use different lib commits.
Perhaps consider purchasing a support contract if people's charity endangers your users.
For non-security updates, they can do the rebuild in a branch (staging) so it's non-blocking, but you will feel the pain on a security update.
In practice, users will apply a config mitigation if available or "graft" executables against an updated lib using system.replaceDependencies, which basically search and replaced the built artifacts to use a different file path.
Still, this requires a somewhat bigger infrastructure
It is a difference of security, because now when there's an issue with (e.g.) OpenSSL, instead of just the OpenSSL distro package team having to worry about it, the MySQL distro package team has to, the Postgres distro package team, Nginx, Apache, cURL, OpenSSH, etc.
All of those teams have to coordinate to release at the same time.
Yes, by "the MySQL package team" I meant the MySQL distro package team. Edited to clarify.
But even if applied to the upstream team that releases from (e.g.) repo.mysql.org or repo.postgres.org it would apply: they're now worrying about all their dependencies as well their own code, instead of just specifying in the .deb or .rpm file "I need libssl3" and letting the OS take care of it.
It would become a huge duplication of effort to keep up on security updates.
At $JOB we have a general policy to always use the distro-supplied version of a package whenever possible, as if we compile it on our own and put it into /usr/local or /opt, we now have to babysit that code for security and such ourselves instead of 'outsourcing' it to the distro.
Ain't nobody got time for that.
Maintaining a secure package of zlib takes linearly more time with more versions of it used.
All distros are manpower limited.
That is a large difference
But more significantly, you're assuming the only software installed is from the distro. This isn't what OP is talking about (unless the distro screwed up their package dependencies to get the not found error discussed.
Outside the distro-provided software, you either have to rebuild them all manually or wait for the proprietary software to do it and ship you a fix.
in a world of SSDs that wear out, the concept of updating the entire install every single time, seems problematic.
“The” main argument? In a world filled with diverse concerns, there isn’t just one argument that makes a decision. Additionally, security is one of those things where practically everything is a trade off. Eg: by having lots of things link against a single shared library, that library becomes a juicy target.
> With a shared zlib, you will have to update only one package.
We are back to efficiency :)
For instance, many modern languages use techniques that are simply incompatible with dynamic dispatch. Some languages like Swift have focused on dynamic dispatch, but mostly because it was a fundamental requirement placed on their development teams by executives.
While there is a place for dynamic dispatch in software, there is also no inherent justification for dynamic dispatch boundaries to be exactly at organizational ones. (For example, there is no inherent justification for the dynamic dispatch boundary to be exactly at the places a binary calls into zlib.)
edit: I guess loading up a .so is more commonly called "dynamic binding". But it is fundamentally dynamic dispatch, ie figuring out what version of a function to call at runtime.
I say this as a casual maintainer of several apps and I'm loathe to manually patch versus upstream any fix.
The problem one usually finds with distro repo packages, is they are usually out of date compared to the upstream – especially if you are running a stable distro release as opposed to the latest bleeding edge. You can get in a situation where you are forced to upload your whole distro to some unstable version which may introduces lots of other issues, just because you need a newer version of some specific package. Upstream binary distributions, Flatpak/etc, generally don't have that issue.
> the firefox package from distro repos enables me to load my own extensions without Mozilla's gatekeeping, the Audacity package from distro repos did not have telemetry enabled that Audacity devs added to their own builds, etc
This is mainly a problem with "commercial open source", where an open source package is simultaneously a commercial product. "Community open source" – where the package is developed in people's spare time as a hobby, or even by commercial developers where the package is just some piece of platform infrastructure not a product in itself, is much less likely to have this kind of problem.
That maximizes your chance of being able to satisfy a particular dependency like libflac.8.so. Sometimes that might not actually be practical to pull in or might involve massively changing a lot of your installed software to satisfy the dependencies, but often it can be a quick easy way to drop in more libraries.
Sometimes libraries don't have a version number on them, so it'll keep being libflac even across major versions. Thats prohibitive because ideally you want to install old version 8 alongside newer version 12. But generally Debian is pretty good about allowing multiple major versions of packages. Here for example is libflac12, on stable and unstable both. https://packages.debian.org/search?keywords=libflac12
Actually, please don't. There's a nontrivial chance that you'll ship a copy with bugs that somebody wants to fix.
Instead, to get ALL the advantages of static linking with none of the downsides, what you should do is:
* dynamically link, shipping a copy of the library alongside the binary
This allows the user to find a newer build of the same-version library. It's also much less likely to be illegal (not that anybody cares about laws).
* use rpath to load the library
Remember: rpath is NOT a security hole in general. The only time that can happen is if it is both a privileged program (setuid, setcap, etc.) and it contains a non-owner-writable path, neither of which is likely to happen for the kind of software people complain about shipping.
* do not use dlopen, please
`dlopen` works well for plugins, but terribly for actually-needed dependencies.
Your problem with "libFlac.8.so is missing" happens with using software not from the distro repos. Feel free to statically link it, or run it via AppImage or Flatpak or Podman or whatever you want that provides the environment it *was* compiled for. Whether the rest of the distro is dynamically linked or not makes no difference to your ability to run this software, so there's no reason to make the rest of the distro worse.
I personally do care about disk usage and memory usage. I also care about using software from distro repos vs Flatpak etc wherever possible, because software in the distro repos is maintained by someone whose values align with me and not the upstream software author. Eg the firefox package from distro repos enables me to load my own extensions without Mozilla's gatekeeping, the Audacity package from distro repos did not have telemetry enabled that Audacity devs added to their own builds, etc.