Author here: this is very much a research project with the purpose of prototyping the idea that caching can help tail latency. Previously, people didn't believe that caches can be used that way and it turns out you really have to rethink how you use the caches.

We build a testbed and replayed production workloads. This is not running in production yet. We're actively looking for new workloads and scenarios to test this at scale!