News for hackers. Stuff that matters.
Re:GitHub is degraded/down by xkcd-sucks on Tuesday April 21, 2020 @12:03PM (#22936170)
In an ideal world, pushing new features would have no impact on stable mature features like browsing files, comment threads, etc
Re:GitHub is degraded/down by grogenaut on Tuesday April 21, 2020 @12:32PM (#22936519)
Internal to Amazon we consider that about 80% of issues/outages/etc are due to changes. This may sound "duh" but this is over 10k plus investigations.

Much if the work is just minimizing the impact of this changes by finding them before customers do.

This includes things like unit, integration testing, canaries, cellular/ zonal / regional deploys, auto rollbacks, multi-hour bakes, auto load tests, and much much monitoring. Not to mention cross team code reviews, game days, ops reviews.

Re:GitHub is degraded/down by duxup on Tuesday April 21, 2020 @12:07PM (#22936214)
Ideal I agree ... and yet the real world is exactly the opposite ;)
Re:GitHub is degraded/down by leetrout on Tuesday April 21, 2020 @12:06PM (#22936209)
That’s part of the micro service promised land, right?
Re:GitHub is degraded/down by cranekam on Tuesday April 21, 2020 @12:19PM (#22936341)
No, I think it's more part of how to run a complex system with a lot of people changing stuff at once. Having good monitoring, kill switches, staged rollout, continuous deployment, and so on are all things that contribute more making a reliable service than how microserviced it is.
Re:GitHub is degraded/down by fastball on Tuesday April 21, 2020 @05:14PM (#22939399)
Too bad this is the real world.
Re:GitHub is degraded/down by lovegoblin on Tuesday April 21, 2020 @12:51PM (#22936707)
Which world is that?