News for hackers. Stuff that matters.
Re:The abysmal state of GNU/Linux and a case against shared ... by ChocolateGod on Monday October 14, 2024 @04:30PM (#41841618)
Correct me if I'm wrong Linux only supports KSM (memory-deduping) between processes when doing it between VMs, as QEMU provides information to the kernel to perform it.
Re:The abysmal state of GNU/Linux and a case against shared ... by yjftsjthsd-h on Monday October 14, 2024 @06:13PM (#41842633)
https://www.kernel.org/doc/html/latest/admin-guide/mm/ksm.ht...

> KSM was originally developed for use with KVM (where it was known as Kernel Shared Memory), to fit more virtual machines into physical memory, by sharing the data common between them. But it can be useful to any application which generates many instances of the same data

Although...

> KSM only operates on those areas of address space which an application has advised to be likely candidates for merging, by using the madvise(2) system call

Re:The abysmal state of GNU/Linux and a case against shared ... by pradn on Monday October 14, 2024 @10:22PM (#41844319)
I wonder if you could just madvise the entire address space. My hunch is that it's for performance reasons only - fewer pages to scan, hash, and de-duplicate.
Re:The abysmal state of GNU/Linux and a case against shared ... by MrDrMcCoy on Tuesday October 15, 2024 @01:26AM (#41845220)
You can. There are custom kernel builds that do this, as well as shim loaders that do that madvise call. However, those are considerably less practical than the recent systemd feature that allows you to do it per unit. If applied to the system scope, it effectively covers the whole system. However, there are caveats:

1. You need a much better newer systemd than your distro likely packages.

2. KSM dedupe scans aren't free. Your system can spend its time doing that scan or it can spend its doing work with duplicate pages. Only relatively idle or highly homogenous systems would be free of the penalty.

3. For applications, especially statically linked ones, the duplicated code is not super likely to fall along page boundaries, thus the actual detectable duplication will be relatively low.

That said, it's still great for densely deploying a high traffic microservice that's more memory than CPU bound.