Re:Roblox October Outage Postmortem
by Karrot_Kream on Thursday January 20, 2022 @08:32PM
(#30018035)
If you're doing a slow rollout, it's not always easy to tell whether the thing you're rolling out is the culprit. I've been on the other side of this outage where we had an outage and suspected a slow change we had been rolling out, especially because we opted something new into it minutes before an incident, only to realize later when the dust settled that it was completely unrelated. When you're running at high scale like Roblox and have lots of monitoring in place and multiple pieces of infrastructure at multiple levels of slow-rollout, outages like this one don't quickly point to a smoking gun.
[ Parent
| on HN ]
There are no comments on this comment yet.