News for hackers. Stuff that matters.
Re:32-bit x86 Position Independent Code – It's That Bad by zvrba on Wednesday April 15, 2015 @02:50AM (#9379267)
This is an oft-repeated mantra, but is it really true? I don't have a linux desktop running here, but try an experiment: start up a desktop environment such as KDE, and write "free -m" and see how much is there under "shared" heading. Without shared libraries, some extra memory corresponding to some multiple of that number would be used.

To calculate exactly how much memory is saved by shared libraries, you'd need to write a kernel module to walk the internal structures describing physical pages and summing reference counts of used pages. Maybe it's already been done?

Re:32-bit x86 Position Independent Code – It's That Bad by bandrami on Wednesday April 15, 2015 @04:47AM (#9379632)
some extra memory corresponding to some multiple of that number would be used

Depends. If I have 7 instances of my terminal emulator loaded (say I hadn't discovered tmux yet or something), the loader can share their .rodata and .text segments, and in many cases it does (YMMV; heuristics apply; void where prohibited; etc.). So a lot of things that are in shared libraries right now might still be only loaded into memory once if their binaries are segmented correctly.

The Plan9 people (Plan9 doesn't do dynamic linking) claim that the memory savings they get from skipping the relocation overhead are greater than the memory hit from the times that the same stuff does get loaded multiple times, though obviously always take self-promotion with a grain of salt.

Use case probably matters -- my servers run few processes to begin with, and it's often a lot of versions of the same process, whereas my laptop runs a ton of very different processes. Same sort of argument that makes me happy with udev on my laptop while also very happy with a static /dev tree on my servers.

Re:32-bit x86 Position Independent Code – It's That Bad by zvrba on Wednesday April 15, 2015 @05:46AM (#9379849)
I agree, sharing will (most probably) still work on segment-level [0]. But this means that if I have two different executables each linking an identical version of, say QT, in its .text segment, those copies will not be shared.

[0] In fact, certainly. Program loading works by mmaping the executable with MAP_SHARED flag into the process's address space, and the VFS takes care to keep all mappings coherent. The simplest way of keeping mappings coherent is to actually share the backing storage between all mappings. It's the foundation of CoW.

Re:32-bit x86 Position Independent Code – It's That Bad by caf on Wednesday April 15, 2015 @05:04AM (#9379702)
You could do that calculation in userspace with a stock kernel, by summing up the sizes of the non-writable mappings of the various library files in /proc/*/maps, then using mincore() to find out how much of the libraries are resident.
Re:32-bit x86 Position Independent Code – It's That Bad by zvrba on Wednesday April 15, 2015 @05:50AM (#9379867)
mincore is a system call and does not take a pid parameter. This means it has to be executed by each process individually, which would require injecting code into each running process and executing it in some way.

Unless there's a tool which makes this extremely simple (maybe Intel's pin?), I believe that writing a kernel module is simpler. The module's init function tallies up the pages and writes out the result into the kernel log. Then the module exits.

Re:32-bit x86 Position Independent Code – It's That Bad by caf on Wednesday April 15, 2015 @06:17AM (#9379949)
You don't need to run mincore() on target pids - you just need to write a tool that opens(O_RDONLY) and mmaps(PROT_READ) each library file, then calls mincore() on each page in the mapping to find out which pages of the library are loaded shared.

The results of mincore() in one process with a shared mapping of a file are enough to tell you how much of that file is loaded shared system-wide.

Re:32-bit x86 Position Independent Code – It's That Bad by zvrba on Thursday April 16, 2015 @03:06AM (#9386330)
Just found out that physical page information is exposed through /proc: https://www.kernel.org/doc/Documentation/vm/pagemap.txt
Re:32-bit x86 Position Independent Code – It's That Bad by zvrba on Wednesday April 15, 2015 @06:34AM (#9379992)
Ah, good point! Though I think I'll do a dlopen on each file to mimic whatever the loader usually does. I'll do it as a weekend-project :)
Re:32-bit x86 Position Independent Code – It's That Bad by caf on Wednesday April 15, 2015 @06:57AM (#9380058)
In case it helps, here's my little utility (which just shows how much of a file is in core):

https://github.com/keaston/fincore