News for hackers. Stuff that matters.
Re:GitHub is going down almost every day around the same time by throwawaaarrgh on Wednesday May 10, 2023 @09:31AM (#35887098)
I would like to see real stats on this. Most people don't properly quantify availability and assume just because a big system goes down a lot that one they run could be more reliable, but it's usually apples and oranges.
Re:GitHub is going down almost every day around the same time by samdcbu on Wednesday May 10, 2023 @09:39AM (#35887198)
From an SLA uptime perspective, it’s also worth considering that you don’t have a whole team working to fix your self-hosted Gitlab server when it goes down. So one outage overnight of your self-hosted server could be more downtime than more frequent, shorter GitHub outages.
Re:GitHub is going down almost every day around the same time by duncan-donuts on Wednesday May 10, 2023 @09:43AM (#35887257)
This is kind of a straw man don’t you think? Really small startups sure but I’m sure many places that self host have an ops team that can and does respond to outages on their systems.
Re:GitHub is going down almost every day around the same time by actuallyalys on Wednesday May 10, 2023 @10:02AM (#35887509)
Your example also suggests another factor: downtime overnight might be less consequential than shorter outages that occur during working hours.
Re:GitHub is going down almost every day around the same time by dec0dedab0de on Wednesday May 10, 2023 @10:28AM (#35887898)
Have you ever troubleshot an outage with a whole team? I am strongly in the belief that any more than two people working on an outage makes it last longer.
Re:GitHub is going down almost every day around the same time by throwawaaarrgh on Wednesday May 10, 2023 @02:18PM (#35891231)
Properly run incident management can have three dozen people involved with no negative impact. You need the incident manager to coordinate, communicate, run interference. You need a clear set of rules for who will have what role and responsibility. Combination of voice and text, multiple chat channels.

But yeah, if you have no plan or organization, too many cooks is detrimental.

Re:GitHub is going down almost every day around the same time by hariharan_uno on Wednesday May 10, 2023 @09:36AM (#35887154)
I think even if the percentage uptime is less on a self hosted one, at least you get more control over when to upgrade systems (perhaps in the evenings / weekends) with less impact to the devs.
Re:GitHub is going down almost every day around the same time by eole666 on Wednesday May 10, 2023 @09:49AM (#35887331)
I've worked using self-hosted Gitlabs for more than 6 years, and they're has been almost no down time, mostly quick updates lasting a few minutes from time to time. Companies tend to fear self-hosted services because tech giants are supposed to be way more reliable and do everything for you, but owning your infrastructure and having people in place who actually know what's going on can be a life saver in many situations.
Re:GitHub is going down almost every day around the same time by throwawaaarrgh on Wednesday May 10, 2023 @02:07PM (#35891055)
Unfortunately this experience doesn't pan out the same for a company:

- A large number of users will increase the likelihood that one of them will encounter a temporary issue; even if problems are intermittent, a single person using the system simply won't notice them, but lots of people will.

- More users tends to mean using more features, more system resources, etc, which increases the likelihood of an issue from either buggy/complex features, or resource exhaustion.

- Updates for a team typically involve being more careful and performing updates on a regular schedule, in order to minimize downtime.

- If a problem does occur and you're not around, the team is stuck until you are available. If you go on vacation or are hit by a bus, they're stuck longer.

- Having more users tends to require things like disaster recovery contingencies, security, etc. Somebody has to do this extra work, which is a cost in time and labor.

Where self hosted shines is in keeping complexity down and changes minimal. GitHub is a giant system, making it more likely for problems to interrupt service. They constantly ship changes, increasing the likelihood of interrupted service. A self hosted option can use the same simple system and version for much longer, only taking security patches until the version is EOL and needs to be upgraded.