News for hackers. Stuff that matters.
Re:How We Knew It Was Time to Leave the Cloud by Rapzid on Saturday November 12, 2016 @04:57PM (#12940437)
This got me curious again about the pluggable storage backends in Git(I assume AWS code commit is using something like this). I've looked at Azures blob storage API in the past and found it incredibly flexible..

Here is an article from a few years ago: http://blog.deveo.com/your-git-repository-in-a-database-plug...

In any case, GitLab is amazing and I can see how it's tempting to believe that GitLab the omnibus package is the core product. However, HOSTED GitLab's core product is GitLab as a SERVICE. That might require designs tailored a bit more for the cloud than simply operating a yoooge fs and calling it a day.

Re:How We Knew It Was Time to Leave the Cloud by solatic on Sunday November 13, 2016 @01:36AM (#12942168)
Gitlab is limited by the fact that they need their hosted product to run the same code as their on-prem product in order to avoid forking the codebase.

If on-prem customers can't get AWS/GCE/Azure SuperFastCloudStorage™, then it can't be part of their codebase.

Re:How We Knew It Was Time to Leave the Cloud by sytse on Sunday November 13, 2016 @06:02AM (#12942796)
Exactly, we want our users to be able to scale with us using open source technologies. Some are at 20k+ users so they are close to needing something like Ceph themselves.
Re:How We Knew It Was Time to Leave the Cloud by Rapzid on Sunday November 13, 2016 @05:48PM (#12946174)
From looking at the comments here, thinking about what I would do/want, and what others have done to scale git GitLab is in the minority of wanting to solve this issue with a Ceph cluster.

> Some are at 20k+ users so they are close to needing something like Ceph

Or they will scale it themselves ala Alibaba: http://www.slideshare.net/MinqiPan/how-we-scaled-git-lab-for... . They appear to have written a libgit2 backend for their object store(among other things).

I don't see a good reason why solutions using different storage backends could not make it into the OSS project. Many companies run their own Swift cluster, which is OSS.

If you're using CephFS and everyone else wants to be using other Cloud storage solutions, that would actually put you at a disconnect with your users and leave room for a competitor with the tools and experience to scale out on Cloud storage to come in offering support. I would at least consider all the opinions in this thread and maybe reach out to that Minqi Pan fellow from Alibaba with questions..

I actually really like GitLab and wish we could be using it at my company; this is why I'm spending so much effort on this topic(and scaling git is interesting). Hopefully my opinions are not out of place.

Re:How We Knew It Was Time to Leave the Cloud by sytse on Sunday November 13, 2016 @06:30PM (#12946415)
Thanks, we're in touch with Minqi Pan I think we all agree the Ceph solution is great if we can make it work.