News for hackers. Stuff that matters.
Re:GitHub Incident by ghusto on Tuesday March 22, 2022 @12:26PM (#30768116)
With something as simple as serving git for your own organisation, when you do get downtime it's going to be a case of restoring from backup and changing DNS. No need for complicated infrastructure, so bringing it back really is that simple.

Worst case is losing commits between going down and backups, but even then it'll be minimal if you set your backup to simply push to a another origin.

I pay for someone else to worry about most services, but git doesn't have to be one of them.

Re:GitHub Incident by munificent on Tuesday March 22, 2022 @12:29PM (#30768150)
The worst case is discovering the hard way that your backup system isn't working and you haven't been diligent about spending the time to periodically test restores.
Re:GitHub Incident by dspillett on Tuesday March 22, 2022 @01:49PM (#30769198)
That's why testing of backups should be at least partially automated. Though you will always find a point where the automation, and automated tests to make sure that is working, and so on, get to the point where making a human responsible for sanity checking is most resource effective.
Re:GitHub Incident by ghusto on Tuesday March 22, 2022 @02:15PM (#30769490)
Pushing to multiple origins is not a hard thing.
Re:GitHub Incident by munificent on Wednesday March 23, 2022 @02:49PM (#30781866)
The risk is less about losing commits, since those tend to live on every devs machines too. It's about losing all of the other data like the issue tracker, comments, etc.
Re:GitHub Incident by thefreeman on Tuesday March 22, 2022 @12:30PM (#30768173)
If you think "serving git" is the only role github is serving for organizations you are probably not the target audience.
Re:GitHub Incident by ghusto on Tuesday March 22, 2022 @02:16PM (#30769498)
I don't. Which is why I was explicit in the problem I'm targeting.