News for hackers. Stuff that matters.
Re:Having fun with Redis Replication between Amazon and Rack... by otterley on Tuesday July 31, 2012 @01:10PM (#4318385)
Of course you'll have to have presence in multiple DCs. I didn't mean to imply that one would suffice.

I'm not sure whether you've analyzed the bandwidth of the replication stream, but my experience is that if you've got a good circuit provider, you should have adequate bandwidth for 30Mbps+ streams, except under extremely unusual conditions, even over the open Internet.

Re:Having fun with Redis Replication between Amazon and Rack... by solso on Tuesday July 31, 2012 @06:33PM (#4319993)
The replication stream is about 20-25 Mbps for the current throughput of 25K-30K req/s, our peak to valley ratio is low.

It seems that we have found this unusual conditions you mention :-) Before compression our replication was often lagging (when below the required bandwidth), which is very dangerous, even if after a minute bandwidth goes well above the threshold.

The case of replication is kind of worst case, it's no good to have an average of 30 Mbps if it stays a substantial fraction of the time below the threshold. For every hours at 75% memory grew 2GB, but even one minute below the threshold is bad... if the replication queue builds up and there is a crash, consistency goes out of the window. So better to over-dimension capacity.

We even did a quick test with Hetzner instead of Rackspace, it was worse. The average over 4 hours was about 15 Mbps.