Re:Do not buy RAM, compress
by RossBencina on Thursday November 19, 2026 @03:33PM
(#50565965)
[delayed]
Compression and decompression of memory is potentially even beneficial for CPU as the memory bandwidth to the CPU is much slower than the CPU itself.
I'm learning about this now. Apparently it's not as simple as comparing decompression speeds vs memory bandwidth.
Compression happens in 16KB pages. But memory reads can be 64-byte or 128-byte lines. So from a compressed page, the CPU cannot directly read a line from memory.
So to read even a single 8-byte line, the kernel will page fault, allocate a fresh 16KB page, decompress the entire 16KB page into it, update the page tables, flush the TLB, then resume execution.
I'm not too familiar with LLM memory access patterns, but since the model files are stored on disk, you wouldn't ever keep them in memory anyway (except caching) and just reread them on demand.