News for hackers. Stuff that matters.
Re:Everything old is new again: memory optimization by wat10000 on Friday March 27, 2026 @10:20AM (#47542965)
Turning these numbers into "memory consumption" gets complicated to the point of being intractable.

The portions that are allocated but not yet used might just be page table entries with no backing memory, making them free. Except for the memory tracking the page table entries. Almost free....

A lot of "image" will be mmapped and clean. Anything you don't actually use from that will be similarly freeish. Anything that's constantly needed will use memory. Except if it's mapped into multiple processes, then it's needed but responsibility is spread out. How do you count an app's memory usage when there's a big chunk of code that needs to sit in RAM as long as any of a dozen processes are running? How do you count code that might be used sometime in the next few minutes or might not be depending on what the user does?

Re:Everything old is new again: memory optimization by gmueckl on Friday March 27, 2026 @03:16PM (#47547031)
This assumes that executable code pages can be shared between processes. I'm skeptical that this is still a notable optimization on modern systems because dynamic linking writes to executable memory to perform relocations in the loaded code. So this would counteract copy on write. And at least with ASLR, the result should be different for each process anyway.
Re:Everything old is new again: memory optimization by cataphract on Friday March 27, 2026 @04:48PM (#47548056)
ld writes to the GOT. The executable segment where .text lives is not written to (it's position independent code in dynamic libraries).

ASLR is not an obstacle -- the same exact code can be mapped into different base addresses in different processes, so they can be backed by the same actual memory.

Re:Everything old is new again: memory optimization by manwe150 on Friday March 27, 2026 @08:45PM (#47550326)
That’s true on most systems (modern or not), but actually never been true on Windows due to PE/COFF format limitations. But also, that system doesn’t/can’t do effective ASLR because of the binary slide being part of the object file spec.
Re:Everything old is new again: memory optimization by gmueckl on Friday March 27, 2026 @08:46PM (#47550336)
I can't reconcile this with the code that GCC generates for accessing global variables. There is no additional indirection there, just a constant 0 address that needs to be replaced later.
Re:Everything old is new again: memory optimization by cataphract on Friday March 27, 2026 @09:32PM (#47550608)
Assuming the symbol is defined in the library, when the static linker runs (ld -- we're not talking ld.so), it will decide whether the global variable is preemptable or not, that is, if it can be resolved to a symbol outside the dso. Generally, by default it is, though this depends on many things -- visibility attributes, linker scripts, -Bsymbolic, etc. If it is, ld will have the final code reach into the GOT. If not, it can just use instruction (PC) relative offsets.
Re:Everything old is new again: memory optimization by gmueckl on Friday March 27, 2026 @09:41PM (#47550666)
I've never observed a (non-LTO) linker exchange instructions. I want to see an example before I can believe this.
Re:Everything old is new again: memory optimization by cataphract on Saturday March 28, 2026 @02:00PM (#47556882)
I'm not sure if you're just trolling, but I'll give the same example I gave before (you can get even wilder simplifications -- called relaxations -- with TLS, since there are 4 levels of generality there). I'm not sure what you meant by "changing isntructions", but in the first case the linker did the fixup indicated by the relocation and in the second reduced the generality of the reference (one less level of indirection by changing mov to lea) because it knew the symbol could not be preempted (more exactly, the R_X86_64_REX_GOTPCRELX relocation allows the linker to do the relaxation if it can determine that it's safe to)

  root@1f0775a74fd7:/tmp# cat a.c
  int glob;
  int main() {
   return glob;
  }
  root@1f0775a74fd7:/tmp# gcc -c a.c -fPIC -o a.o
  root@1f0775a74fd7:/tmp# objdump --disassemble=main a.o
  
  a.o:     file format elf64-x86-64
  
  
  Disassembly of section .text:
  
  0000000000000000 <main>:
     0: f3 0f 1e fa           endbr64
     4: 55                    push   %rbp
     5: 48 89 e5              mov    %rsp,%rbp
     8: 48 8b 05 00 00 00 00  mov    0x0(%rip),%rax        # f <main+0xf>
     f: 8b 00                 mov    (%rax),%eax
    11: 5d                    pop    %rbp
    12: c3                    ret
  root@1f0775a74fd7:/tmp# readelf -rW a.o | grep glob
  000000000000000b  000000030000002a R_X86_64_REX_GOTPCRELX 0000000000000000 glob - 4
  root@1f0775a74fd7:/tmp# gcc -shared -o a.so a.o
  root@1f0775a74fd7:/tmp# objdump --disassemble=main a.so
  (...)
  00000000000010f9 <main>:
      10f9: f3 0f 1e fa           endbr64
      10fd: 55                    push   %rbp
      10fe: 48 89 e5              mov    %rsp,%rbp
      1101: 48 8b 05 b8 2e 00 00  mov    0x2eb8(%rip),%rax        # 3fc0 <glob-0x4c>
      1108: 8b 00                 mov    (%rax),%eax
      110a: 5d                    pop    %rbp
      110b: c3                    ret
  (...)
  root@1f0775a74fd7:/tmp# readelf -r a.so | grep glob
  000000003fc0  000600000006 R_X86_64_GLOB_DAT 000000000000400c glob + 0
  root@1f0775a74fd7:/tmp# gcc -shared -Wl,-Bsymbolic -o a.symb.so a.o
  root@1f0775a74fd7:/tmp# readelf -r a.symb.so | grep glob
  root@1f0775a74fd7:/tmp# objdump --disassemble=main a.symb.so
  (...)
  Disassembly of section .text:
  
  00000000000010f9 <main>:
      10f9: f3 0f 1e fa           endbr64
      10fd: 55                    push   %rbp
      10fe: 48 89 e5              mov    %rsp,%rbp
      1101: 48 8d 05 04 2f 00 00  lea    0x2f04(%rip),%rax        # 400c <glob>
      1108: 8b 00                 mov    (%rax),%eax
      110a: 5d                    pop    %rbp
      110b: c3                    ret
  (...)
Re:Everything old is new again: memory optimization by gmueckl on Friday March 27, 2026 @09:38PM (#47550647)
OK, I spent a few additional minutes digging into this. It's been too long since I looked at those mechanisms. Turns out my brain was stuck in pre-PIE world.

Global variables in PIC shared libraries are really weird: the shared library's variable is placed into the main program image data segment and the relocation is happening in the shared library, which means that there is an indirection generated in the library's machine code.

Re:Everything old is new again: memory optimization by wat10000 on Friday March 27, 2026 @08:49PM (#47550349)
Are you looking at the code before or after the static linker runs?
Re:Everything old is new again: memory optimization by wat10000 on Friday March 27, 2026 @05:19PM (#47548401)
Dynamic linking doesn't have to write to code. I'm not familiar with other platforms, but on macOS, relocations are all in data, and any code that needs a relocation will indirect through non-code pages. I assume it's similar on other OSes.

This optimization is essential. A typical process maps in hundreds of megabytes of code from the OS. There are hundreds of processes running at any given time. Eyeballing the numbers on an older Mac I have here (a newer one would surely be worse) I'd need maybe 50GB of RAM just to hold the code of all the running processes if the pages couldn't be shared.