Also, sibling comments argue that kernel samepage merging can help avoid the bloat of static linking. But here what you argue will make every copy of the shared libraries oh-so-slightly-different and therefore prevent KSM from working at all. Really, no one is thinking this through very well. Even distributions that do static linking in all but name (such as NixOS) do still technically use dynamic linking for the disk space and memory savings.
Abstractions are a thing in computer science. Abstracting at the shared library layer makes as much sense as abstracting in the RPC layer (which your software is most likely going to be obligated to do) or abstracting at the ISA level (which your software IS obligated to do). Your software has as many chances to break from a library change as it does from a display driver change or from a screen resolution change or from a processor upgrade. Why the first would bloat the "testing matrix" but not the later is over me, and already shows a bias against dynamic linking: you assume library developers are incapable of keeping an ABI but that the CPU designers are. (Anecdotally, as a CPU designer, I would rather trust the library developers..)
Many modern software development paradigms are simply not compatible with ABIs or dynamic binding. Dynamic binding also likely means you're leaving a bunch of performance on the table, since inlining across libraries isn't an option.
You'd be surprised, specially when I'm thinking 30 year old software. Again, usually I can patch around it thanks to dynamic linking...
> Many modern software development paradigms are simply not compatible with ABIs or dynamic binding
This is nonsense.
> Dynamic binding also likely means you're leaving a bunch of performance on the table, since inlining across libraries isn't an option.
Again, why set the goalpost here and not say ISA level or any other abstraction layer? I could literally make the same argument to any of these levels (e.g. "you are leaving a bunch of performance on the table" by not specializing your ISA to your software). How much are you really leaving? And how much would you pay if you remove the abstraction? What are the actual pros/cons?
The answer to each of these questions is specific to the circumstances. You have to decide based on general principles (what do you value?), the specific facts, and ultimately judgment.
I think in some cases (e.g kernel or libc) using dynamic binding generally makes sense, but I happen to think forcing shared library use has many more costs than benefits.
You're absolutely right that everyone should ask these questions, though. I work at Oxide where we did ask these questions, and decided that to provide a high-quality cloud-like experience we need much tighter coupling between our components than is generally available to the public. So. for example, we don't use a BIOS or UEFI—we have our own firmware that is geared towards loading exactly the OS we ship.
> This is nonsense.
Monomorphization like in C++ or Rust doesn't work with dynamic binding. C macros and header-only libraries don't work with dynamic binding either.
The fact of the matter is outside of GPU drivers and poorly designed GUI applications, that's an extreme statistical outlier.
For the record, the 1.1GB executable I'm thinking about is a popular _terminal-only_ simulation package. No GUI.
I did a little math using a common GTK application as an example: nautilus. Using `ldd` and several shell utilities, if you were to statically link it it clocks in at around 80MB in binary size, but that would be worst case without LTO.
FWIW I think graphics stacks are one of the few places where dynamic linking is still highly relevant.
Just watch how these become used in every other app over the next few years.
And yes, the fact that you won't use those apps does not make it easier for the rest of us.
If you've got 10-15+ consumers of a shared library or want to do plugins/hot-code reloading and have a solid versioning story by all means vend a DSO. If you don't however I would strongly recommend trying to keep all dependencies static and letting LTO/LTCG do its thing.
It would be true that this save memory if applications did not increase their memory requirements over time, but the fast is that they do, and the rate at which they increase their memory use seems to be dictated not by how much memory they intrinsically need but how much is available to them.
There are notable exceptions. AI models, Image manipulation programs etc. do actually require enough memory to store the relevant data.
On the other hand I have used a machine where the volume control sitting in the system tray used almost 2% of the system RAM.
Static linking enables the cause of memory use to be more clearly identified, That enables people to see who is wasting resources. When people can see who is wasting resources, there is a higher incentive to not waste them.
If we pay money to someone to audit our books, we are more likely to achieve more within our budget.
I don't know which OSs do this, but I know hypervisors certainly do this across multiple VMs.
It's unlikely for the OS to effectively deduplicate memory pages from statically linked libraries across different applications.
I guess much of this is why its hard to use shared libraries in the first place.
> KSM was originally developed for use with KVM (where it was known as Kernel Shared Memory), to fit more virtual machines into physical memory, by sharing the data common between them. But it can be useful to any application which generates many instances of the same data
Although...
> KSM only operates on those areas of address space which an application has advised to be likely candidates for merging, by using the madvise(2) system call
1. You need a much better newer systemd than your distro likely packages.
2. KSM dedupe scans aren't free. Your system can spend its time doing that scan or it can spend its doing work with duplicate pages. Only relatively idle or highly homogenous systems would be free of the penalty.
3. For applications, especially statically linked ones, the duplicated code is not super likely to fall along page boundaries, thus the actual detectable duplication will be relatively low.
That said, it's still great for densely deploying a high traffic microservice that's more memory than CPU bound.
https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsp...
I assume there's something optimising that away but I'm not well versed.
400 megabytes of memory usage is probably worth more than 400 megabytes of storage. It may not be a make-or break thing on it's own, but it's one of the reasons Linux can run on lower-end devices.