News for hackers. Stuff that matters.
Re:Do not buy RAM, compress by hobo123 on Thursday November 19, 2026 @03:03PM (#50565519)
Since you need to apply CPU time to compress memory, and also CPU before you can map pages back into process memory (by uncompressing), this is only useful for not frequently used memory, like swap only a bit faster.

I'm not too familiar with LLM memory access patterns, but since the model files are stored on disk, you wouldn't ever keep them in memory anyway (except caching) and just reread them on demand.

Re:Do not buy RAM, compress by RossBencina on Thursday November 19, 2026 @03:33PM (#50565965)
[delayed]
Re:Do not buy RAM, compress by origini on Thursday November 19, 2026 @04:08PM (#50566388)
Compression and decompression of memory is potentially even beneficial for CPU as the memory bandwidth to the CPU is much slower than the CPU itself.
Re:Do not buy RAM, compress by acchow on Thursday November 19, 2026 @06:04PM (#50567817)
I'm learning about this now. Apparently it's not as simple as comparing decompression speeds vs memory bandwidth.

Compression happens in 16KB pages. But memory reads can be 64-byte or 128-byte lines. So from a compressed page, the CPU cannot directly read a line from memory.

So to read even a single 8-byte line, the kernel will page fault, allocate a fresh 16KB page, decompress the entire 16KB page into it, update the page tables, flush the TLB, then resume execution.