FYI
https://en.oxforddictionaries.com/definition/err_on_the_side...
Just in case you ever use that phrase in critical correspondence. Better to err on the side of caution.
Re:Data Loss at GitLab
by leesalminen on Wednesday February 01, 2017 @08:00PM
(#13547100)
Ah, thanks! It didn't feel right but was too lazy to check. Will definitely remember this for next time.
Recently, our Redis cluster failed and both master & slave host machines rebooted.
This caused a latency spike in our app, from roughly a 70ms to 400ms response time for less than 10 minutes.
We posted to status within 60 seconds and posted 3 updates within those 10 minutes.
The next day, a new customer (who hadn't gone live with the app yet) cancelled their subscription because the app was "not reliable".
I guess my point is that there is a balance to strike. Our customers are not tech-savvy in any way and treat any small issue as the end of the world. Maybe there's no need to freak people out for a minutes-long latency spike.