AWS us-east-1 outage

Re:AWS us-east-1 outage by sakopov on Tuesday December 07, 2021 @10:55AM (#29473862)

Those five 9s don't come easy. Sometimes you have to prop them up :)

[ Parent | on HN ]

Re:AWS us-east-1 outage by jjoonathan on Tuesday December 07, 2021 @11:17AM (#29474246)

I wonder how often outages really happen. The official page is nonsense, of course, and we only collectively notice when the outage is big enough that lots of us are affected. On AWS, I see about a 3:1 ratio of "bump in the night" outages (quickly resolved, little corroboration) to mega too-big-to-hide outages. Does that mirror others' experiences?

[ Parent | on HN ]

Re:AWS us-east-1 outage by Spivak on Tuesday December 07, 2021 @11:33AM (#29474532)

If you count any time AWS is having a problem that impacts our production workloads then I think it's about 5:1. Dealing with "AWS is down" outages are easy because I can just sit back and grab some popcorn, it's the "dammit I know this is AWS's fault" outages that are a PITA because you count yourself lucky to even get a report in your personalized dashboard.

[ Parent | on HN ]

Re:AWS us-east-1 outage by jjoonathan on Tuesday December 07, 2021 @12:18PM (#29475314)

Yep.

Random aside: any chance you are related to the Calculus on Manifolds Spivak?

[ Parent | on HN ]

Re:AWS us-east-1 outage by clh1126 on Tuesday December 07, 2021 @12:30PM (#29475501)

I had to log in to say, that one of my favorite quotes of all time I found in Calculus on Manifolds.

He says that any good theorem is worth generalizing, and I've generalized that to any life rule.

[ Parent | on HN ]

Re:AWS us-east-1 outage by Spivak on Tuesday December 07, 2021 @12:27PM (#29475448)

Nope, just a fan. It was the book that pioneered my love of math.

[ Parent | on HN ]

Re:AWS us-east-1 outage by kylemh on Tuesday December 07, 2021 @11:20AM (#29474301)

https://aws.amazon.com/compute/sla/

looks like only four 9's

[ Parent | on HN ]

Re:AWS us-east-1 outage by dotancohen on Tuesday December 07, 2021 @12:22PM (#29475385)

  > looks like only four 9's

That's why the Germans are such good engineers.

  Did the drives fail? Nein.
  Did the CPU overheat? Nein.
  Did the power get cut? Nein.
  Did the network go down? Nein.

That's "four neins" right there.

[ Parent | on HN ]

Re:AWS us-east-1 outage by JoelMcCracken on Tuesday December 07, 2021 @11:38AM (#29474633)

Every time someone asks to update the status page, managers say "nein"

[ Parent | on HN ]

Re:AWS us-east-1 outage by 1-6 on Tuesday December 07, 2021 @11:38AM (#29474620)

It’s hard to measure what five-9 is because you have to wait around until a 0.00001 occurs. Incentivizing post-mortems are absolutely critical in this case.

[ Parent | on HN ]

Re:AWS us-east-1 outage by notinty on Tuesday December 07, 2021 @02:02PM (#29476755)

It's 0.001; the first 2 9's count.

  5N  = 99.999%
  3N  = 99.9%
  1N5 = 95%

5N is <43m12s downtime per month.

[ Parent | on HN ]

Re:AWS us-east-1 outage by 1-6 on Tuesday December 07, 2021 @03:13PM (#29477603)

I considered writing it as a percent but then decided against using it and moving the decimal instead. But good info for clarification.

[ Parent | on HN ]