Here's the Feb 2012 thread with Facebook's Joshua Redstone regarding their experiments with git:
http://comments.gmane.org/gmane.comp.version-control.git/189...
In August 2012, they hinted they were near a solution:
http://www.quora.com/Facebook-Engineering/Has-Facebook-solve...
...and that teaser was just today updated with the remark, "Our Mercurial team has just posted the goods! Our problems are solved, mainly by not using Git."
:/
Fundamentally I like how this cleans up the design flaw of requiring history everywhere. 99% of the time when I clone or pull or diff or whatever I only care about HEAD. Why should I be forced to pull or store GBs of history or even MBs of metadata I can't use? Why not make leaving this data on the server optional? I can see how the decision to push history everywhere was made for simplicity but it doesn't reflect real world usage and clearly isn't scalable. Let's hope these history-option patches continue to be developed and make their way upstream. They certainly have my vote, not just as options but as defaults.
I mean, they run into a resource allocation problem for an application that is essentially a glorified web view, and they think "how can I hack up the VM to bypass this limitation?" Sounds like insanity to me.
Each individual step seems resonable and sane thing -- but when you look at the result of all those steps -- you stand back and just stare in awe at the horror you have created... it hits you just how far you have drifted.
In a recent meeting, we had a corporate redirection a bit -- and a co-worker simply asked the question "Is this a sane approach to what our actual problem is?" -- glad to be working with a crew who askes those kinds of questions.
Just because their solution is complex doesn't mean they're using the technology wrong. They set out their requirements and they met all of them. All their engineers are more efficient now and all this complexity happens behind the scenes.
I'm curious - do you know of any such better examples, or is this merely a theoretical "I feel like we could do better" statement?
Basically though, repo is good for a single project ("Android", as a collective entity) but a company looking at working with many repos will want something designed with handling many unrelated projects at once. (For example, all the repos for "Android" and all the repos or "ChromeOS" should be able to live alongside each other in this system without any developer hassle, even though they may not have anything to do with each other).
Also repo is really only one part of the solution a company should look at. Properly done, the sort of system that I am talking about is really several systems that are all basically on the same page. Your build servers should understand how to get the code they need to build something, your internal code search/browsing tools should understand where code lives, etc. Re
Furthermore, allowing different types of repos to exist alongside each other is a good idea. That way each team can individually decide if they want to use Mercurial, Git, or even SVN.. To my knowledge, the repo script is only good for many git repositories.
(my email is listed in my HN profile)
I personally think, if you want to take Git to the next level, you'll have to implement something like UCM ClearCase activities and projects. Commits should not be bound to a single repository and it should be very easy for people to say I want to create a product that uses starting points from branches x, y and z without having to think about what repo they belong to.
I have a couple examples that shows how my product de-emphasizes the repository. In the following example, you can see my Commits Finder tool, which lets you search for commits by branches.
http://ny.testdrive.gitsense.com/index?#pid=13&cid=20&trail=
As you can see, the search results shows commits from 10 different branches from 7 different repositories. My other example is my GitHub Pulls Finder tool which lets you search for pull requests by branches.
http://ny.testdrive.gitsense.com/index?#pid=24&cid=29&trail=
Here you can see pull requests from 6 different branches from 6 different repos.
Since my solution is read-only, it simplified things, but I don't think creating a nice layer on top of sub-modules would be be that technically challenging. And it's the direction I would personally go if I wanted to make Git more enterprise friendly.
This is the fundamental problem Facebook (and Google, Amazon, etc) are all trying to solve: how to share code company-wide. Sub-repos and making your tools hack around the repos doesn't solve that problem.
Say one aspect of a package suddenly starts to grow quite fast and take on a life of its own, and you would like to split it into its own dedicated package (and therefore, it's own repo). How do you do that without loosing the history of those files?
It's possible with git.. but it isn't exactly what I would call straightforward. With a one-repo system you simply move those files to their new location, just like any other file move/rename.
I don't know how you did it, but I found `git subtree split` to be an easy solution for extracting a directory into its own repo:
https://github.com/git/git/blob/master/contrib/subtree/git-s...
(It's a shame git-subtree it's still in contrib/, should really be enabled by default. It can also work as git-submodule replacement in some cases, by the way.)
Either isn't as nice as `p4 move ...` though.
Keeping a giant repository on the other hand makes it very easy to do stupid things and end up with tightly coupled code. You don't want that to be the path of least resistance.
You can get all your many repos in one place, with all of your internal systems handling them nicely, but moving one file from one repo to another, in a way that history remains clear, is something of an unsolved problem.
>Git of course hits a brick wall much sooner, somewhere around single-digit gigabytes of data (depending heavily on the average size of every object in the DAG), even ignoring throughput.
if you use Git the same way you'd use Perforce or Mercurial - one central server handling everything ... you'd better stay with Perforce or Mercurial. Corporate dev process is usually built around notion of "central" repo (and they equate it with "master") - just replacing Perforce server with Git server wouldn't change anything.
The problem is "how do you grow even further?" There isn't a perfect solution to that (yet), but I think the only existing workable (although absolutely imperfect) solution is splitting your codebase into many small repos.
Thankfully very few companies need to worry about this problem right now. A single Mercurial repo is apparently scaling for Facebook currently, and few companies will manage to stress Perforce.
For all of the source code in the world that is source-controlled, it exists in separate repos. It doesn't exist in one repo, and where there are inter-project dependencies, it is the dependency consumer's responsibility to keep abreast of the changes in the dependency and integrate them as necessary. It is self-organizing.
So, if Google and Facebook have grown beyond what can be done with a single repo, either they are not being mindful of their engineering practices (which puts the public at significant risk), or they are approaching a scale that is more similar to "world scale" than it is "corporate scale".
Which I would also say is putting the public at significant risk.
that is the problem - single logical repo in Perforce/Mercurial does mean single physical repo. Which obviously causes issues at very large scale. Even at normal enterprise scale :)
Git solves it through kind of distributed scaling where many operations can be performed on local physical repos without ever hitting the central. With centralized solutions, like Perforce, Mercurial, etc... most of the operations are performed on central server and your ability to scale vertically hits the ceiling pretty soon.
Compare the simplest case - each dev updating his local workspace/repo would hit the central server in Perforce where is with Git you can (and normally would have) a small set of downstream repos which the updates would be propagated/distributed through. You can branch left and right in Git in your local and team's repos without "master" repo and "central" server involved (while all your branch activity in Perforce is happening right on the central server in the central repo). The same happens for synchronizing your work inside the team - no need to hit the central. Etc...
my bad. 5+ years since i worked with Mercurial. Digging deep into painful memories, my Mercurial PTSD from that time is absence of in-repo branching - need to clone which is a killer for very large repo we had and no partial commits - again a killer aggravated by the above mentioned absence of in-repo branches. Both issues made working with large repo unreasonably and unnecessary hard.
These days, the standard practice is probably to use bookmarks, which are more or less like Git branches.
No. You have the choice to use either named branches or bookmarks, and which one you choose is a matter of your workflow. Note that even a Git-like workflow does not necessarily require the use of bookmarks.
Mercurial is still designed under the assumption that people will use named branches and the Git-like aspects of bookmarks exist to ease transition from Git: http://www.selenic.com/pipermail/mercurial/2014-January/0464...
Personally, I use Fossil-like private branches [1] in order to facilitate local work that I may still want to reorganize before pushing it to a shared server; I use bookmarks solely for local tags.
[1] Emulated by putting a named branch in the secret phase. I actually have a simple extension to automatically make all branches that start with a dot (e.g, ".tempstuff") secret.
What do you mean by partial commit? I can guess but I would rather know. :-)
I think you're incorrectly putting mercurial and perforce in the same bucket. Mercurial is a lot more like Git than perforce in that it's a DVCS. Facebook made it scale by using a lot of centralized caching, but most operations (diff, looking through history, branching, etc) can be done locally.
But webkit, and chromium, as well as other GIANT projects which as far as I know are larger then Facebook seem to work fine on Git.
webkit and chromium have about 6 or 7 million lines of code and < 200,000 commits.
Facebook 6 months ago had 62 million lines of code and > 1000 commits a day.
http://www.quora.com/Facebook-Engineering/How-many-lines-of-...
Also, there's a difference between those open source projects using large repos with git and Facebook wanting to increase developer efficiency. Developers sitting around waiting for a rebase doesn't really pay off, where as an open source project can get away with it.
<old-comment> Facebook's problem is that they were trying to scale with Git improperly. With conventional CVS[sic] systems like Perforce, you can scale a single repo nearly as large as any company will need. Emphasis on that nearly. At a certain point, with a large enough codebase (and, critically, enough throughput) you start to realize that you are about to hit a brick wall. With perforce, this starts to manifest with service brownouts.
With perforce, you can reasonably expect to run into this brick wall somewhere in the neighborhood of terabytes of metadata and dozens of transactions per second. That changes depending on what sort of beastly hardware you are willing to throw at your version control team.
Git of course hits a brick wall much sooner, somewhere around single-digit gigabytes of data (depending heavily on the average size of every object in the DAG), even ignoring throughput.
Perforce is probably good enough for Facebook in the present, but if you are a company that large and if you are forward looking, it becomes apparent that with existing version control technology, "one repo per company" is not a long term solution. Even "one repo per department".
You can split it even further, but what you realize is that you are developing infrastructure that allows you to use many repos (for instance your build servers and internal code search/browsing tools will now need to understand that concept) but you are losing many of the benefits of Perforce. While you are in the process of adapting your infrastructure to wrap its mind around many repositories, it makes sense to allow dev teams to really take advantage of this splitting. Develop infrastructure that allows Perforce and git repos to coexist in the company, allowing dev teams to spin up new git repos for their every project at will. Done properly, git allows you to create massively scalable systems that you can count on supporting your companies needs for the foreseeable future.
Smooth migration, migration that does not disrupt development, takes months (assuming the right initial conditions), so it is best to recognize the problem and start early, before service becomes disrupted.
If I understand Facebook's situation currently, they are still in the "try to make Mercurial scale" stage of denial, burning developer time and effort to push back that first brick wall (the same one that git hits, though mercurial hits it after git hits it but before perforce hits it...)
Here is a google presentation about extreme scaling with Perforce: http://www.perforce.com/sites/default/files/still-all-one-se...
An example of building multi-repo infrastructure for large projects with git is Android's repo: http://en.wikipedia.org/wiki/Repo_(script) Repo is just one example though; other, better, solutions are very possible.
</old-comment>
I guess the TL;DR here is that I think it is great that Facebook is making a single Mercurial repository work for their purposes right now, but I think they are kidding themselves if they think that is a long term solution. They are doing a pretty good job of making Mercurial scale like Perforce can scale, but that will only work for them for so long.
(In the above comment I talk about building a scalable system with git ("with" git, rather than making git itself scale), but the same can of course be done with mercurial instead of git. I don't mean this to be a comment suggesting that they should use git instead of mercurial.)