It's a hard problem; we are basically building a distributed database from scratch to solve it. You have to pick a position on the DB-design trade-off curve where you can store state with low latency, consistently, where the state is very ephemeral (long running handlers aside, generally it should only live for a few minutes). But more than anything, the 'trick' is storing service state, execution state (ie the journal), and communication between services all in the same distributed log - that means you can commit all 3 atomically