Re:Major AWS Outage Happening
by junon on Monday October 20, 2025 @12:47PM
(#45646020)
Yep. And their internal comms were on the same server if memory serves. They were also down.
[ Parent
| on HN ]
Re:Major AWS Outage Happening
by simplyluke on Monday October 20, 2025 @02:28PM
(#45647357)
I was there at the time, for anyone outside of the core networking teams it was functionally a snow day. I had my manager's phone number, and basically established that everyone was in the same boat and went to the park.
Core services teams had backup communication systems in place prior to that though. IIRC it was a private IRC on separate infra specifically for that type of scenario.
Thanks for the correction, that sounds right. I thought I had remembered IRC but wasn't sure.
Re:Major AWS Outage Happening
by prmoustache on Monday October 20, 2025 @05:45PM
(#45649821)
I remember working for a company who insisted all teams had to usr whatever corp instant messaging/chat app but our sysadmin+network team maintained a jabber server + a bunch of core documentation synchronized on a vps in a totally different infrastructure just in case and sure enough there was that a day it came handy.
Re:Major AWS Outage Happening
by elchananHaas on Monday October 20, 2025 @06:06PM
(#45650032)
AWS, for the ultimate backup, relies on a phone call bridge on the public phone network.
Ah, but have they verified how far down the turtles go, and has that changed since they verified it?
In the mid-2000s most of the conference call traffic started leaving copper T1s and going onto fiber and/or SIP switches managed by Level3, Global Crossing, Qwest, etc. Those companies combined over time into Century Link which was then rebranded Lumen.
As of last October, Lumen is now starting to integrate more closely with AWS, managing their network with AWS's AI: https://convergedigest.com/lumen-expands-fiber-network-to-su...
"Oh what a tangled web we weave..."