News for hackers. Stuff that matters.
Re:Microsoft blames monster email outage on software defect ... by EGreg on Monday November 16, 2026 @11:52AM (#50519724)
In software servers this is called the “thundering herd” problem. It is why clients are coded do not just exponential backoff but random offsets too.
Re:Microsoft blames monster email outage on software defect ... by chinathrow on Monday November 16, 2026 @12:22PM (#50520100)
It's also a well known problem, so no excuse for that.
Re:Microsoft blames monster email outage on software defect ... by bravetraveler on Monday November 16, 2026 @03:51PM (#50523075)
Business continuity/disaster recovery should also be well-known/defined, agreed. No excuses.

I did SRE at another firm, not Google like the parent but a similar level. We tested these procedures twice a year and every time we added a region.

Re:Microsoft blames monster email outage on software defect ... by dijit on Monday November 16, 2026 @04:38PM (#50523675)
a well known problem with more than a few known fixes, one of which is load shedding your upstream based on protocol aware load balancrs.

I agree: no excuse.

Re:Microsoft blames monster email outage on software defect ... by VirusNewbie on Monday November 16, 2026 @05:49PM (#50524614)
Not only did they not handle thundering herd, it happened across all regions.
Re:Microsoft blames monster email outage on software defect ... by olalonde on Monday November 16, 2026 @05:52PM (#50524656)
Also, circuit breaker pattern.