News for hackers. Stuff that matters.
Re:Analyzing the codebase of Caffeine: a high performance ca... by masklinn on Sunday February 02, 2025 @10:26AM (#42909214)
Wouldn’t one hit wonders still be an issue? They might get evicted relatively fast anyway but assuming an LRU each will still take a cache entry until they go through the entire thing and finally get evicted.

Although if that’s your concern you can probably just add a smaller admission cache in front of the main cache, possibly with a promotion memory.

Re:Analyzing the codebase of Caffeine: a high performance ca... by JanecekPetr on Monday February 03, 2025 @03:12AM (#42916038)
That's kind of the idea of Caffeine, it has admission buffers, and it adapts automatically between LRU and LFU. The original algorithm is called Windiw TinyLFU (design https://github.com/ben-manes/caffeine/wiki/Design), see it in action e.g. here: https://github.com/ben-manes/caffeine/wiki/Efficiency
Re:Analyzing the codebase of Caffeine: a high performance ca... by masklinn on Monday February 03, 2025 @05:25AM (#42916823)
I know that, but I’m not replying to a comment about caffeine, rather the opposite.
Re:Analyzing the codebase of Caffeine: a high performance ca... by NovaX on Monday February 03, 2025 @11:34PM (#42928007)
I think the idea is that the cache is so large that hot data won't be forced out by one-hit wonders. In this 2017 talk [1], the speaker says that Twitter's SLA depends on having a 99.9% hit rate. It is very common to have extremely over provisioned remote caching tier for popular sites. That makes eviction not as important and reducing their operational costs comes by purging expired data more proactively. Hence, memcached switched from away from relying on its LRU to discard expired entries to using a sweeper. Caffeine's approach, a timing wheel, was considered but dormando felt it was too much of an internal change for memcached and the sweeper could serve multiple purposes.

[1] https://www.youtube.com/watch?v=kxMKnx__uso