News for hackers. Stuff that matters.
Re:I built Foyer: a Rust hybrid cache that slashes S3 latency by jitl on Saturday September 27, 2025 @01:29PM (#45397742)
I wonder to what degree it’s actually necessary to explicitly manage memory spilling to disk like this. Want unified interface over non durable memory + disk? There already is one: memory with swap.

Materialize.com switched from explicit disk cache management to “just use swap” and saw substantial performance improvement. https://materialize.com/blog/scaling-beyond-memory/

To get good performance from this strategy your memory layout already needs to be optimized to pages boundaries etc to have good sympathy for underlying swap system, and you can explicitly force pages to stay in memory with a few syscalls.

Re:I built Foyer: a Rust hybrid cache that slashes S3 latency by hamandcheese on Saturday September 27, 2025 @01:57PM (#45397980)
> There already is one: memory with swap.

Actually, there's already two. The other one: just read from disk (and let OS manage caches).

Re:I built Foyer: a Rust hybrid cache that slashes S3 latency by cowsandmilk on Saturday September 27, 2025 @05:40PM (#45399559)
I prefer this abstraction as it is more widely supported (I’ve had to deploy to hosts that intentionally kill when you swap) and results in development assuming that the access may be at disk speed. When you rely on swap, I often see developers assuming everything is accessible at memory speed and then surprised when swap causes sudden degradation.
Re:I built Foyer: a Rust hybrid cache that slashes S3 latency by rockwotj on Saturday September 27, 2025 @08:42PM (#45400725)
Yeah I mean if you have a latency sensitive workload, you don’t want page faults and swapping to give hidden latency spikes - it kills your P99 latencies
Re:I built Foyer: a Rust hybrid cache that slashes S3 latency by jitl on Saturday September 27, 2025 @09:23PM (#45400947)
Yeah but you can pin important pages in RAM with mlock family of syscalls and friends like move_pages for explicit page management. This is what materialize does as far as I understand it.
Re:I built Foyer: a Rust hybrid cache that slashes S3 latency by rockwotj on Saturday October 04, 2025 @11:59PM (#45478739)
That seems to be contradictory, as they talk about using it to spill to disk in a simpler way. You can’t swap with mlock?
Re:I built Foyer: a Rust hybrid cache that slashes S3 latency by rockwotj on Wednesday October 08, 2025 @04:43AM (#45513677)
Also move_pages is about numa nodes not spilling to swap
Re:I built Foyer: a Rust hybrid cache that slashes S3 latency by shikhar on Saturday September 27, 2025 @01:34PM (#45397778)
Re:I built Foyer: a Rust hybrid cache that slashes S3 latency by MrCroxx on Sunday September 28, 2025 @01:37AM (#45402024)
Hi, foyer's author here. Page cache and swap are indeed good general proposed strategies and continuously evoluting. There are several reasons why foyer manages memory and disk control by itself rather than directly utilizing these mechanisms:

1. Leverage asynchronous capabilities: Foyer exposes async interfaces so that while waiting for IO and other operations, the worker can still perform other tasks, thereby increasing overall throughput. If swap is used, a page fault will cause synchronous waiting, blocking the worker thread and resulting in performance degradation.

2. Fine-grained control: As a dedicated cache system, foyer has a better understanding than a general proposed system like the operating system's page cache of which data should be cached and which should not. This is also why foyer has supported direct I/O to avoid duplication of abilities with the page cache since day one. Foyer can use its own strategies to know earlier when data should be cached or evicted.