So for the canonical use case of session state in a load balanced server farm, is the session state being replicated across every single one of these in-process caches? Or, if a request is routed to a server without a copy of the session state, does the in-process cache have to go find a peer server with a copy of the session state?

This seems like an easy “solution” to show how it speeds up the happy path while not fully addressing the real-world tradeoffs.

Stateless application servers come and go without a care in the world. You can reason very simply about the cost and effects of bringing them up and down.

Add in a KV store with sharding, replication, a discovery protocol, a heartbeat protocol, a sync and recovery protocol, etc... and put it all in-process on every application server?

Have fun monitoring and debugging this.

I much prefer the 3-tier system of a local RAM cache, a Redis cache, and a persistence layer. As much state as possible is idempotent so you can cache it locally as well as in Redis. The load balancer can best-effort route back to the same app server, and your KV lookups automatically check local RAM first. But the central KV store handles replication and is easy to monitor and scale out if needed.