Re:An update on Sunday’s service disruption
by thundergolfer on Tuesday June 04, 2019 @01:32AM
(#20092194)
How do this code-review of cmds work? Does the command get saved to a file, and then that file is reviewed like regular source-code, and then when it is approved the cmd is copy-pasted back to the terminal and run?
That above seems pretty clunky, so it's very likely not what happens.
Re:An update on Sunday’s service disruption
by zedgerman on Tuesday June 04, 2019 @02:24AM
(#20092428)
I’ve seen scripts get checked in and deployed just like you would a new service (code). Same Code Review process and same release pipeline.
In this particular case, commands that were run on a Production machine were by-design limited to what they can do and affect (mostly just the physical host they’re run on or a few hosts in the logical group of hosts they belong to).
See a similar outage in S3 from 2 years ago - https://aws.amazon.com/message/41926/