News for hackers. Stuff that matters.
Re:Cache made consistent: Meta’s cache invalidation solution by uvdn7 on Wednesday June 08, 2022 @04:03PM (#31673226)
> isn't the typical bug related to cache invalidation that a service forgets to contact the cache server for invalidation?

That's not the case based on our experience at FB. At-least-once delivery is a solved problem basically.

But you are absolutely right that if there's an issue in the invalidation delivery, it's possible that polaris won't receive the event as well. Polaris actually supports a separate event stream (all the way from client initiated writes) to cover this case.

Re:Cache made consistent: Meta’s cache invalidation solution by benlivengood on Wednesday June 08, 2022 @04:27PM (#31673530)
It helps that TAO is a write through cache so clients can't really forget to invalidate, correct? If someone were to directly write to MySQL shards there would be stale data ~indefinitely.

I'm assuming the versioning is what ensures proper write, invalidation, and fetch ordering so that e.g. slow mysql writes/replication don't cause remote TAO clusters to receive an invalidation message and then read a stale value from a replica?

Re:Cache made consistent: Meta’s cache invalidation solution by uvdn7 on Wednesday June 08, 2022 @04:41PM (#31673742)
> It helps that TAO is a write through cache so clients can't really forget to invalidate, correct?

That's not the case.

> If someone were to directly write to MySQL shards there would be stale data

For the sake of this discussion, you can assume this is the setup, and everything should still stand.

> I'm assuming the versioning is what ensures proper write, invalidation, and fetch ordering so that e.g. slow mysql writes/replication don't cause remote TAO clusters to receive an invalidation message and then read a stale value from a replica?

You should join us :p This is getting into the details of our cache invalidation protocol and our cache consistency model. Maybe another post on another day!