Re:Building resilient services at Prime Video with chaos eng...
by ses1984 on Monday August 24, 2020 @03:12PM
(#24264267)
Your own instances can have an unexpected CPU spike.
Even if you're running on bare metal I find it hard to believe you don't have a layer with short timeouts between your front and backend.
Re:Building resilient services at Prime Video with chaos eng...
by fxtentacle on Monday August 24, 2020 @04:16PM
(#24265025)
Why would I? I have redundant 1GBit LAN cables between front end, back end, and database servers.
Re:Building resilient services at Prime Video with chaos eng...
by ses1984 on Monday August 24, 2020 @06:17PM
(#24266165)
Because it's bad ux for your users to see a spinning loading icon forever.
Re:Building resilient services at Prime Video with chaos eng...
by fxtentacle on Monday August 24, 2020 @06:36PM
(#24266324)
And a timeout error would be better?
Well, with the 2x example, my app might get from a 1s P99 to a 2s P99 which feels slow, but is still doable. Again, those timeouts are usually introduced by cloud infrastructure. For example, if you use nginx outside of Heroku, it won't have a 30s timeout for file downloads.