News for hackers. Stuff that matters.
Re:Former Japan PM Shinzo Abe Unconscious After Shooting; Ma... by mjr00 on Friday July 08, 2022 @12:44AM (#32023291)
There are both models at Amazon. Individual teams have their own on-call; if for example an RDS instance is continually failing to boot, someone on-call (edit: from the RDS team) will get paged to look at it.

I believe order drops are looked at by dedicated reliability team, because it's such a significant problem. But they don't watch graphs because... why would they? They set up the machines to page them when metrics aren't within an expected range. Which is what happened.

Re:Former Japan PM Shinzo Abe Unconscious After Shooting; Ma... by majormajor on Friday July 08, 2022 @12:48AM (#32023321)
That makes a lot of sense.

The ops teams I've worked with watched graphs because (a) sometimes you see interesting changes or trends before they turn into actionable alerts and people can look into them/figure them out before shit's on fire, and (b) it makes it look more like you're doing something when everything is smooth sailing and not actively on fire, solving the "wait why are we paying for this if it doesn't take us long to respond to incidents anyway" naive-meddlesome-exec problem.