Re:Summary of the Amazon S3 Service Disruption
by umanwizard on Thursday March 02, 2017 @12:31PM
(#13776087)
One of the positive things about Amazon's culture is that they heavily emphasize blaming broken processes, not blaming people. I doubt the person involved will have any negative consequences beyond embarrassment.
Re:Summary of the Amazon S3 Service Disruption
by erikbye on Thursday March 02, 2017 @01:03PM
(#13776483)
I would be horrified if I learned that Amazon or any other company of such size in any way castigates employees for such very human errs. The guilt (don't beat yourself up) he or she likely feels is bad enough.
Anyway, to me this firstly sounds like a "tool" or command that was too powerful with not enough safeguards. Who knows, the command might even be ambiguous.
Re:Summary of the Amazon S3 Service Disruption
by Raphmedia on Thursday March 02, 2017 @12:52PM
(#13776339)
Can you imagine the feeling? I once sent a debug email to our mailing list and felt terrible for days. Imagine bringing down the internet...
Re:Summary of the Amazon S3 Service Disruption
by detaro on Thursday March 02, 2017 @12:34PM
(#13776123)
> At 9:37AM PST, an >>> authorized <<< S3 team member [...]
typos happen, if the system didn't stop them that's a design issue or accepted risk.
Re:Summary of the Amazon S3 Service Disruption
by apetresc on Thursday March 02, 2017 @12:30PM
(#13776079)
The article specifically said "authorized".
Re:Summary of the Amazon S3 Service Disruption
by danesparza on Thursday March 02, 2017 @12:30PM
(#13776083)
How do you figure? From the blog post:
" At 9:37AM PST, an authorized S3 team member using an established playbook executed a command which was intended to remove a small number of servers for one of the S3 subsystems that is used by the S3 billing process. Unfortunately, one of the inputs to the command was entered incorrectly and a larger set of servers was removed than intended."
Re:Summary of the Amazon S3 Service Disruption
by jessaustin on Thursday March 02, 2017 @06:49PM
(#13779493)
Haha it's funny that a simple misread of "authorized" as "unauthorized" got this comment downvoted to oblivion...
Re:Summary of the Amazon S3 Service Disruption
by beberlei on Thursday March 02, 2017 @01:28PM
(#13776746)
The post never once blames human error, but always specifies that the tools were the problem.
Re:Summary of the Amazon S3 Service Disruption
by rocgf on Thursday March 02, 2017 @12:32PM
(#13776115)
I think the article says he was authorized? Either way, at worst, he got fired.
Re:Summary of the Amazon S3 Service Disruption
by jmelloy on Thursday March 02, 2017 @01:14PM
(#13776594)
I assume if they were unauthorized they would be fired, but since they were authorized shit happens.
"Fire you? I just spent $10 million training you!"
Re:Summary of the Amazon S3 Service Disruption
by beamatronic on Thursday March 02, 2017 @01:45PM
(#13776912)
Exactly - this person is now the expert on the error, and they can now impart this valuable knowledge to the organization.
EDIT: derp, my bad, I read as "unauthorized" which was "authorized".