Re:Cache made consistent: Meta’s cache invalidation solution
by nixpulvis on Wednesday June 08, 2022 @07:00PM
(#31675543)
Correct me if I'm wrong, but don't we have general solutions that ensure correctness if you allow for enough time? Wouldn't performance be a critical aspect of the claim that this is "no longer a hard problem"? How about the CAP theorem?
[ Parent
| on HN ]
Re:Cache made consistent: Meta’s cache invalidation solution
by uvdn7 on Wednesday June 08, 2022 @07:04PM
(#31675584)
The "hard problem" defined here is more about the engineering side. The analogy I have is Paxos the protocol and Google's Paxos Made Live paper. We do have many cache coherency protocols that are provably correct (using TLA+ if you will); but making them actually consistent in production is a completely different story; and a hard problem for an invalidation-based cache.
Re:Cache made consistent: Meta’s cache invalidation solution
by gnomeduck on Thursday June 09, 2022 @06:09AM
(#31679418)
Can you speak more on why ‘making them actually consistent in production is a completely different story’?
Curious because I’ve been learning TLA+ recently and interested to more know about cases where an algorithm has been proven but actual an implementation of it fails.
Re:Cache made consistent: Meta’s cache invalidation solution
by uvdn7 on Thursday June 09, 2022 @08:44AM
(#31680505)
Let try to put this as concisely as possible. When you put an algorithm in code, when rubber hits the road, you have to make certain assumptions of how things work, eg how events are ordered, how fsync works, what kind of failure scenarios you are expecting, etc. More likely than not, the reality will be a little different. No to measure just innocent bugs in the code.
My favorite example is Paxos. Its algorithm fits on a single slide. But it’s notoriously hard to make it actually work correctly in production.