News for hackers. Stuff that matters.
Re:Microsoft blames monster email outage on software defect ... by VirusNewbie on Monday November 16, 2026 @05:53PM (#50524677)
>- it took 15 hours to deploy the fix (not good, but at this scale maybe).

I've definitely been in similar scale war rooms and 15 hours is not acceptable.

Re:Microsoft blames monster email outage on software defect ... by mrheosuper on Monday November 16, 2026 @09:36PM (#50526707)
Can you share more, it's not like we have billions-user affected bug everymonth.
Re:Microsoft blames monster email outage on software defect ... by VirusNewbie on Monday November 16, 2026 @11:20PM (#50527396)
sure, it's not that longer outages can't happen, but it's very peculiar that the outage was global, and that mitigation took that long. I would have expected fast mitigation for the most populous zones or regions.

The thing with plain outages vs. say, partial degradation of services, you can roll out fast because you're already at worst case scenario *[1].

Here's an interesting one to compare to that I'm very familiar with:

https://status.cloud.google.com/incidents/xVSEV3kVaJBmS7SZbn...

Another really bad one I did not have any involvement in but studied extensively post hoc:

https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1S...

*[1] - I suppose data loss is worse but let's assume most services are stateless.

Re:Microsoft blames monster email outage on software defect ... by tremon on Tuesday November 17, 2026 @12:27PM (#50534928)
What do you mean, "not acceptable"? Given the situation, I think 15 hours is defensible. Where do you suggest putting the blame for the implementation and deployment of the fix?

What's not acceptable to me is a bug of this impact going undetected for 6+ months in production.