Summary of the Amazon S3 Service Disruption

Re:Summary of the Amazon S3 Service Disruption by mulmen on Thursday March 02, 2017 @06:49PM (#13779492)

Agreed, especially regarding the culture but isn't this pretty much the same explanation they gave a few years ago when something similar happened?

I seem to recall an EC2 or S3 outage a few years ago that boiled down to an engineer pushing out a patch that broke an entire region when it was supposed to be a phased deployment.

I could be mis-remembering that but it's important that these lessons be applied across the whole company (at least AWS) so it would be a bigger mark against AWS if this is a result of similar tooling to what caused a previous outage.

[ Parent | on HN ]

Re:Summary of the Amazon S3 Service Disruption by cjbprime on Thursday March 02, 2017 @08:22PM (#13780004)

Pretty sure that one was a Microsoft Azure outage.

(Source: am a self-identified post-mortems connoisseur. :)

[ Parent | on HN ]

Re:Summary of the Amazon S3 Service Disruption by evilDagmar on Thursday March 02, 2017 @10:13PM (#13780473)

Not a bad plan. If you don't make enough mistakes on your own, ya gotta learn from the mistakes of others as a preventative.

[ Parent | on HN ]

Re:Summary of the Amazon S3 Service Disruption by sah2ed on Friday March 03, 2017 @05:37AM (#13781956)

Do you by chance keep a public log of your postmortem collection :)?

[ Parent | on HN ]

Re:Summary of the Amazon S3 Service Disruption by cjbprime on Friday March 03, 2017 @12:22PM (#13784384)

I don't, but danluu does! https://github.com/danluu/post-mortems

[ Parent | on HN ]

Re:Summary of the Amazon S3 Service Disruption by bobzimuta on Friday March 03, 2017 @09:09PM (#13788718)

Yeah an EC2 engineer switched over traffic to a backup network connection that had significantly less bandwidth, triggering cascading failures.

[ Parent | on HN ]