Re:AWS is down, but here's why the sky is falling
by justinsb on Thursday April 21, 2011 @04:00PM
(#2472125)
It is possible. I think it's exceptionally unlikely.
The one bit of information we have suggests that the root cause was a networking issue, which suggests SPOF.
Re:AWS is down, but here's why the sky is falling
by leoc on Thursday April 21, 2011 @04:27PM
(#2472227)
If I had to, I'd guess that AWS's messaging/monitoring/control infrastructure is likely to be the SPOF, as in the 2008 outage: http://status.aws.amazon.com/s3-20080720.html It's an obvious weak spot in the independence/isolation of AWS' nodes, and it would seem to be the one most likely to cause failure to reach across more than one AZ. (Apart from a stampede from one, affected AZ to others perhaps.)
As far as I know we've heard nothing to the contrary from Amazon - it's totally possible that multiple AZs happened to fail independently/coincidentally. Perhaps it was simultaneous equipment failure? Or maybe one AZ failed and a sufficient number of people attempted to "fail over" to another AZ causing a chain reaction of failure?