News for hackers. Stuff that matters.
Re:GitHub incident 2022-03-23 by raffraffraff on Wednesday March 23, 2022 @03:10PM (#30782093)
Yuck. Honestly, restarting a database to fix a major outage sounds like "we have no idea what we're doing"
Re:GitHub incident 2022-03-23 by blueplanet200 on Wednesday March 23, 2022 @07:45PM (#30785004)
It sounds like "they don't know why it's going down." I've worked with plenty super competent people that have taken time to root cause incidents.

Guide to incidents: Step 1: Stop the bleeding Step 2: Prevent it in the future

Doing Step 1 doesn't make you incompetent.

Re:GitHub incident 2022-03-23 by raffraffraff on Thursday March 24, 2022 @01:05PM (#30792602)
I'm not a DBA, and maybe you're not a DBA either, so this question goes to DBAs who may be reading: aren't you always better off killing the bad queries instead of rebooting the whole box, if that's an option? (ie: aside from times when the entire host is screwed, load per core is >50, metrics aren't getting out, you can't ssh in etc)
Re:GitHub incident 2022-03-23 by bpicolo on Wednesday March 23, 2022 @04:10PM (#30782724)
Sporadic database performance issues can certainly make you feel that way. They are definitely not trivially debugged at scale
Re:GitHub incident 2022-03-23 by vimda on Wednesday March 23, 2022 @04:57PM (#30783263)
Would you rather it stay down while they spend a day debugging it?
Re:GitHub incident 2022-03-23 by paulryanrogers on Wednesday March 23, 2022 @05:47PM (#30783886)
If that means it won't be down every morning in my time zone then yes.
Re:GitHub incident 2022-03-23 by seanw444 on Wednesday March 23, 2022 @07:31PM (#30784898)
As long as it's announced in advance so that users/customers can plan ahead, I don't see why not.
Re:GitHub incident 2022-03-23 by karmakaze on Wednesday March 23, 2022 @07:04PM (#30784696)
They could use multiple writer hosts and rollover the restarts. MySQL has had GTIDs since 5.6 and replication groups rather than writer-replicas since some 5.7.x version.