Re:AWS services to avoid
by manigandham on Sunday May 10, 2020 @05:16PM
(#23136275)
How are you doing active-active?
piggybacked off Dynamodb in AWS for now unfortunately; so its more "the entire system is active-active, not just the redis bit". Reads always go to Redis in every region (no read-through), writes go to dynamodb (in global table config). Then there's a worker in each region consuming the dynamo stream and keeping it's region's redis in sync
Worker is a DIY stream consumer implementation. The stream -> lambda integration works, but its super limited (and ~300ms slower to get new events on average)
* writes are multiplexed to redis and dynamo in the origin region.
Re:AWS services to avoid
by manigandham on Monday May 11, 2020 @07:19PM
(#23148485)
Why is the origin region Redis not also a reader of the replication stream? Just for latency?
yep. in this specific use case there can be pretty fast reads after a write, and dynamo stream latency is not great even intra-region.
Via the lambda integration you should probably expect ~1000ms p99 latency intra-region, and ~2100ms p99 from, e.g, east-1 to west-2. Shave maybe 200-300ms off of those numbers if you do a DIY stream consumer without the aws provided lambda connector.
Favorite datastore to date. The only thing I wish for frequently is native global change-data-capture without doing something manually with streams (I've thought about writing a replication protocol consumer, but the way resync's are handled is a little problematic)