News for hackers. Stuff that matters.
Re:Data Loss at GitLab by praneshp on Wednesday February 01, 2017 @09:26PM (#13547564)
HN can be funny sometimes. GitHub got a lot of hate about a year ago just for not releasing new features. GitLab cost everyone a day because their backup/ops practices were silly, and everyone loves them more.

I've screwed up before, and I sympathize/empathize with their ops folks, but this should make us think about plan B in case something like this happens again.

Re:Data Loss at GitLab by deckar01 on Wednesday February 01, 2017 @10:01PM (#13547726)
> GitLab cost everyone a day

GitLab isn't popular because of the stability of its cloud platform. It's popular because you can install your own instance for free practically anywhere with minimal effort.

I run GitLab CE on a box in my server closet for projects that involve livelihoods.

Re:Data Loss at GitLab by praneshp on Wednesday February 01, 2017 @11:03PM (#13548007)
I know (because I use it and also host it at home, and like it). The point of my comment was HN's reaction.
Re:Data Loss at GitLab by Spare_account on Thursday February 02, 2017 @07:43AM (#13550103)
Remember that HN is not a single-minded mob, during each incident a particular group was vocal on a given subject but that group may not have been the same individuals who were vocal each time.
Re:Data Loss at GitLab by praneshp on Thursday February 02, 2017 @11:14AM (#13551592)
Fair point, thank you.
Re:Data Loss at GitLab by ergo14 on Thursday February 02, 2017 @03:44AM (#13549057)
You can also look at RhodeCode CE - supports git/mercurial/svn and has a really nice installer you can use to keep things up to date.
Re:Data Loss at GitLab by thewhitetulip on Wednesday February 01, 2017 @11:20PM (#13548078)
>It's popular because you can install your own instance for free practically anywhere with minimal effort.

If that is the reason why you use Gitlab, then why not try gitea or gogs? gogs is written in Go and provides a docker image or a drop in binary

Re:Data Loss at GitLab by pritambarhate on Wednesday February 01, 2017 @11:44PM (#13548173)
I have been running Gitlab CE for around three years now and I am very happy with it. The interface it good. The access management works very good. And the best part? Keeping it updated is very easy on a ubuntu box. Just a couple of very easy commands.

I am so happy with Gitlab that I didn't even install Gogs even once to give it a try, though I know about it since it's initial days. For me Gitlab CE just works. Unless Gitea shows a 10x feature I am very unlikely to shift away from Gitlab CE.

Re:Data Loss at GitLab by problems on Thursday February 02, 2017 @01:45AM (#13548625)
GitLab is a lot more feature packed than Gitea/Gogs. Gogs is lightweight, good for personal projects, but if you're looking for something to deploy company wide with integrated everything, Gitlab is the way to go.
Re:Data Loss at GitLab by sdesol on Thursday February 02, 2017 @03:25AM (#13548997)
I think this is changing at a pretty fast pace ... well for Gitea anyways. It also looks like Gitea is lighting a fire under Gogs, as they appear to be iterating at a faster pace as well. Here's a very quick breakdown of what's going on.

Activity for the last 160 days. There were 175 commits to gogs and 720 commits to gitea.

https://gitsense.com/gogs-gitea/commits-160days.png

Activity for the last 60 days. There were 109 commits to gogs and 262 to gitea.

https://gitsense.com/gogs-gitea/commits-60days.png

https://gitsense.com/gogs-gitea/changes-60days.png

https://gitsense.com/gogs-gitea/changes-files-60days.png

The options and vendors directory are unique to Gitea and they account for a lot of the changes within the last 60 days. I was told the vendors directory is used to store dependencies but I don't know what the options directory is used for. And as the following shows, they account for a lot of the files touched, in the last 60 days.

https://gitsense.com/gogs-gitea/changes-options-vendor-60day...

Based on what I've read on Hacker News, the developer behind Gogs, tends to merge in changes in spurts, so it's hard to tell if this recent flurry of activity is a spurt or not. In this 365 days of activity, you can see the 3 spurts for Gogs so far.

https://gitsense.com/gogs-gitea/commits-365days.png

Regardless of whether or not Gogs will continue to develop at an increased rated, it looks like Gitea will.

Re:Data Loss at GitLab by E6300 on Thursday February 02, 2017 @12:12AM (#13548278)
Gogs is pretty sweet. After hearing about it here I installed it on my old home server with constrained memory because I couldn't meet Gitlab's (IMO) obscene memory requirements. It's perfect for my needs as a hobbyist and I even use it as a secondary Git server from work. It's also way, way faster than a self-hosted Gitlab or any public Git server, so even though my current server has more than enough memory to run Gitlab, I still use Gogs.

The only (minor) issue I've had with it was when I tried to push the OpenCV repository to it, just for the hell of it, on a heavily constrained VM (Debian with 256 MiB of RAM). The poor thing just couldn't handle it without crashing until I upped the memory to a couple gigabytes.

Re:Data Loss at GitLab by tscs37 on Thursday February 02, 2017 @04:19AM (#13549190)
It might be related to git and not Gogs/Gitea itself.

I've found Gitea to handle larger repos fine after running the following:

  git config --global core.packedGitWindowSize 16m
  git config --global core.packedGitLimit 64m
  git config --global pack.windowMemory 64m
  git config --global pack.packSizeLimit 64m
  git config --global pack.thread 1
  git config --global pack.deltaCacheSize 1m
This reduces the memory used by git during certain operations.
Re:Data Loss at GitLab by juskrey on Wednesday February 01, 2017 @10:48PM (#13547941)
The one who erred (and recovered) once is more valuable than the one who never made mistakes.
Re:Data Loss at GitLab by bbcbasic on Wednesday February 01, 2017 @10:57PM (#13547984)
There is no basis for that assertion.
Re:Data Loss at GitLab by juskrey on Thursday February 02, 2017 @12:42AM (#13548395)
This is basic risk management. What matters is error survival, not absence of errors. Moreover, total absence of errors means accumulating dangerous risk. See NN Taleb, for example.
Re:Data Loss at GitLab by bbcbasic on Thursday February 02, 2017 @07:16AM (#13549983)
Sounds defeatist to me. Especially when we are not talking about literal moonshots, but backup and restore procedures. The IT equivalent of seatbelts.
Re:Data Loss at GitLab by throwaway2048 on Thursday February 02, 2017 @09:49AM (#13550841)
I think at this point, recognising that software and software deployments invariably have bugs/issues is just realistic. The only setups that are free of them, are setups where the problems havent been discovered yet.
Re:Data Loss at GitLab by shakna on Thursday February 02, 2017 @12:44AM (#13548401)
A well known library like curl, has had CVEs.

A not known library that also addresses the same circumstances as curl might have none listed.

Which would you prefer to use?

Re:Data Loss at GitLab by bbcbasic on Thursday February 02, 2017 @07:13AM (#13549976)
That's a cross purpose argument
Re:Data Loss at GitLab by mondoshawan on Thursday February 02, 2017 @07:33AM (#13550066)
The one with the CVEs because it has had eyes on the source enough to generate them. The devil you know and all that.
Re:Data Loss at GitLab by shakna on Thursday February 02, 2017 @07:39AM (#13550090)
Exactly. They've made mistakes, and can recover from them.

The other hasn't had that chance.

Thus, try the first, as it is better tested in the world.

Re:Data Loss at GitLab by ysavir on Thursday February 02, 2017 @10:00AM (#13550919)
That's very different, though. I trust the company/organization that had to fire an employee for a horrible mistake. They are going to be very careful with who they hire or let near the code in the future, and had much more to lose than a single person.

But I don't trust the employee that was shown forgiveness for a horrible mistake. Some might learn to not repeat what they had done, while others learn that they can get away with things through the magical power of phrasing the situation in a positive light. And some might be mistake-makes-for-life. Not all people come out the other side stronger.

Re:Data Loss at GitLab by astrostl on Thursday February 02, 2017 @01:53PM (#13553316)
I see this play out with regularity in the board game hobby.

Manufacturer A: perfect product. Manufacturer B: omits some pieces, expedites replacements.

People will love and extol the amazing service virtues of B, not A.

Re:Data Loss at GitLab by vollmond on Wednesday February 01, 2017 @09:53PM (#13547690)
There's another reasonable takeaway: GitLab will probably now prepare for this sort of thing better than the average competitor who hasn't been bitten by this.
Re:Data Loss at GitLab by ryanplant-au on Wednesday February 01, 2017 @10:31PM (#13547868)
Borges: "In Alexandria, it has been said that the only persons incapable of a sin are those who have already committed it and repented."
Re:Data Loss at GitLab by sytse on Thursday February 02, 2017 @04:37AM (#13549289)
We'll certainly learn from this, post-mortem write-up issue is in https://gitlab.com/gitlab-com/www-gitlab-com/issues/1108
Re:Data Loss at GitLab by antocv on Thursday February 02, 2017 @04:55AM (#13549361)
Every corporation ever has screwed up like GitLab if not more.

They are just better at hiding it, smoothing it over and lying.

Re:Data Loss at GitLab by marklgr on Thursday February 02, 2017 @02:40AM (#13548831)
That's Internet culture for you: being lovable trumps pretty much anything else.
Re:Data Loss at GitLab by jopsen on Wednesday February 01, 2017 @09:58PM (#13547716)
Nah, no need to think about backup now.

Stuff like this only happens once :) haha,

To be fair, I suspect gitlab will a adopt a 'never-again' policy towards making sure their backups work.

Re:Data Loss at GitLab by yAnonymous on Thursday February 02, 2017 @03:56AM (#13549109)
>GitLab cost everyone a day

That's just nonsense. In most cases, the local repo should be more than enough to continue work.

Re:Data Loss at GitLab by praneshp on Thursday February 02, 2017 @11:36AM (#13551830)
Only for development right? If you were someone that handled issues for the team, or depended on webhooks triggering builds, you were still blocked.

I'm not trying to skewer Gitlab, I host an instance at home. I've also nuked 30% of the ports on our openstack cluster and screwed up everyone's day. I admire the transparency. I just wanted to call out HN's reaction, vs way smaller (technical) issues involving GitHub. But someone did point out downthread that HN's not a hivemind, so there is that.

Re:Data Loss at GitLab by mattcoles on Thursday February 02, 2017 @06:46AM (#13549829)
The actual problem was with their DB, not the repo storage so I'm not sure what the actual fallout was, but people lost things like issues/merge requests, so I can see how workflow would be interrupted.
Re:Data Loss at GitLab by seanp2k2 on Thursday February 02, 2017 @04:32AM (#13549259)
Yeah, kind of defeats the whole D part of DVCS when you centrally-host everything. GitHub is great for creating an open-source watering hole and becoming a de-facto standard for open source project hosting, but bad for mission-critical stuff if you're relying on something like that to be 100% available, especially when using a technology which is designed to be distributed.
Re:Data Loss at GitLab by Floegipoky on Thursday February 02, 2017 @11:23AM (#13551693)
It would have been really cool if GitHub had fully bought into the distributed model and built the wiki/issue tracker/etc such that they were included in the repository, so that they were also decentralized.
Re:Data Loss at GitLab by fictioncircle on Thursday February 02, 2017 @02:53AM (#13548882)
The "Plan B" is to have a local Gitlab instance and use the mirror feature to mirror it to Gitlab.com and/or vice versa.

You should always have 2+ production nodes in case one goes down.