News for hackers. Stuff that matters.

A comment on Google Is 2B Lines of Code, All in One Place.

Re:Google Is 2B Lines of Code, All in One Place by lighthawk on Wednesday September 16, 2015 @01:38PM (#10228373)
"The two internet giants (Google and Facebook) are working on an open source version control system that anyone can use to juggle code on a massive scale. It’s based on an existing system called Mercurial. “We’re attempting to see if we can scale Mercurial to the size of the Google repository,” Potvin says, indicating that Google is working hand-in-hand with programming guru Bryan O’Sullivan and others who help oversee coding work at Facebook."

Why Mercurial instead of Git?

[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by urda on Wednesday September 16, 2015 @01:42PM (#10228399)
Because Google and Facebook are using Mercurial over Git internally.

Edit: And for those that are just shocked that git isn't the answer.

Facebook: https://code.facebook.com/posts/218678814984400/scaling-merc...

Google: http://www.primordia.com/blog/2010/01/23/why-google-uses-mer...

[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by Lewisham on Wednesday September 16, 2015 @01:53PM (#10228480)
Well, Piper conforms to the Perforce API-ish, and Android and Chrome are both on Git.

Mercurial was pushed internally as being the "better" (for some dimension of better) between it and Git back in 2010, but I think even the most hardline Mercurial fans have realized that in order to meet developers in the middle in 2015, we need to use Git for our open-source releases. We have a large investment in Gerrit [1] and Github [2] now.

So the Mercurial comment is probably entirely based on scaling and replacement for the Piper Perforce API, rather than anything externally facing.

[1] https://www.gerritcodereview.com/ [2] https://github.com/google

[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by cap_theorem on Thursday September 17, 2015 @08:06PM (#10236822)
Though both Android and Chromium still build in a way more similar to that for a monolithic repo. They use repo and depot_tools, respectively, as abstractions on top of Git in order to clone and manage all their Git repositories together as if they were a single large repository.
[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by cmrdporcupine on Wednesday September 16, 2015 @02:09PM (#10228579)
I don't think I'm leaking anything by saying that that article about Google is incorrect. Mercurial isn't used at Google, not anywhere I've seen. Probably that article has something to do with the now-defunct Google Code site, which did in fact use Mercurial.
[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by jordigh on Wednesday September 16, 2015 @02:14PM (#10228611)
Google devs are working (with some help from Facebook) on getting hg into Google, but they're not there yet. Look at all the work that Google has been putting into hg. It's not purely out of the goodness of their hearts:

https://selenic.com/hg/[email protected]&revcount=200

I always see a lot of Google and Facebook at the Mercurial sprints.

[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by cmrdporcupine on Wednesday September 16, 2015 @04:05PM (#10229449)
I can't comment on what Google might be doing in the future with RCS. Well, I can't comment but also don't know. I personally like hg.

But if I were to take a poll of the informed and opinionated engineers sitting around me, they would almost all likely take git over hg.

In any case, git is in actual widespread use _now_ at Google, for Android and Chrome, and other open source stuff (some of which was moved recently from code.google.com to github).

[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by spectral321 on Wednesday September 16, 2015 @02:27PM (#10228709)
Google isn't using Mercurial at any significant scale, but there is work being done to see if it can be made to work as a client for Piper.
[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by jsolson on Wednesday September 16, 2015 @05:08PM (#10229883)
There may be Mercurial plans for the future, but to date I've never encountered it at Google. Lots of people use git, though, including folks outside of Chrome and Android who are working against Piper.
[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by thatoneguy on Wednesday September 16, 2015 @02:32PM (#10228743)
I was at Google for a long time and it was almost entirely Perforce (some Googlers used git) until it was Piper (which is very similar to Perforce). L&S used CVS at Stanford for the original Backrub/Google code and for some time after the establishment of Google, Inc.
[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by farresito on Wednesday September 16, 2015 @03:22PM (#10229113)
Is Piper a wrapper around Perforce or built from scratch? What is it exactly, if you don't mind me asking?
[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by thatoneguy on Wednesday September 16, 2015 @07:51PM (#10230767)
It was built from scratch because it was designed as a replacement for Perforce because Perforce is expensive and the G used wrappers around it, anyways.

The commands are still basically the same as Google's wrappers around Perforce and the learning curve was non-existent for someone who was used to using Perforce at Google.

[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by DannyBee on Wednesday September 16, 2015 @03:17PM (#10229080)
There are also a massive number of git repos internal to googl e as well.

One thing mentioned in the paper, but not mentioned here, is that there are teams that live in GIT, such as android and chrome, but they are not monolithic.

[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by foobar2020 on Wednesday September 16, 2015 @01:43PM (#10228407)
How do you know that?
[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by skj on Wednesday September 16, 2015 @09:24PM (#10231044)
He/she doesn't. It's false.
[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by harveynick on Thursday September 17, 2015 @06:48AM (#10232582)
That's not true of Google. At least, not on any of the teams I've worked on. I use a Git wrapper around Piper.
[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by sghi on Wednesday September 16, 2015 @01:43PM (#10228408)
I believe that Facebook ran into problems scaling Git, outlined here - https://code.facebook.com/posts/218678814984400/scaling-merc...
[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by rakoo on Wednesday September 16, 2015 @02:21PM (#10228659)
It looks to me like mercurial is easier to hack on than Git (written in clean Python vs written in C), and in their initial stage both mercurial and git weren't up to the task.
[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by georgemcbay on Wednesday September 16, 2015 @03:28PM (#10229158)
I've used git on repos far less than 2 billion lines of code and it was kind of a nightmare to clone and do basic local checkouts and normal everyday stuff. Out of the box, git's design is not well suited to large repos.

I know there are various workarounds for dealing with large repos but having had some experience using git on big-ish projects I can certainly understand some possible reasons why it wasn't their first choice given the size of their codebase.

[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by cynicalkane on Wednesday September 16, 2015 @05:06PM (#10229864)
I don't work for FB, but word through the grapevine is that FB wanted to use Git, but the Git folks refused to talk to FB about making it scale. By contrast, hg was happy to have them as users and contributors.
[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by grouseway on Wednesday September 16, 2015 @01:55PM (#10228487)
Why not IBM? Why not Microsoft? Is that what the HN crowd has turned into? You now have to justify the non mainstream solution like a scared mega-corp career drone who will only go with the safest bet?

Git isn't so awesome that it's inconceivable that people would be willing to use something else.

[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by scrollaway on Wednesday September 16, 2015 @02:22PM (#10228671)
> Is that what the HN crowd has turned into?

It's a legitimate question, damn it. Especially if the chosen solution is not mainstream.

You don't pick non-mainstream solutions "just because". You do pick mainstream solutions "just because". As in, "just because" they are tried and true, "just because" it's familiar to devs, etc.

So let me ask the reverse question: Is that what the HN crowd [sic] has turned into? You can no longer ask the reasons behind your tech choices?

[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by bigtunacan on Wednesday September 16, 2015 @02:21PM (#10228649)
Personally; I wondered the same thing as the parent, why not Git? For me, this thought was not in a, "How could they possibly not choose Git!?", but more along the lines of, "There are a lot of smart people working at Google; I wonder what technical aspects of Mercurial might make it more suited to such a large code repository than Git?"
[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by malkia on Wednesday September 16, 2015 @03:24PM (#10229134)
I'm at Google now, but would share my opinion of Perforce while I was at Treyarch - Perforce is simply very easy for anyone to use - from software engineer, to game designer, scripter, artist, producer, technical writer, linguist, translator, etc. You can announce your changes by simply saying - it's submitted in CL: 12345678 - grab it and you are done. Eases the communication.

Git and HG are not. Not that they can't be learned, but the learning curve is much higher.

Piper and CitC (mentioned in Rachel Potvik's videO) are even more advanced, I can work from my desktop machine, then go and continue from my laptop, or open and edit files directly from critique in an internal web-based editor.

Key to all this - make it accessible through the web! Anyone can work then from almost any machine. But at the same time the sources is accessible in the file system - even more awesome - so i can use emacs, and whatever other tools are there.

[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by lighthawk on Wednesday September 16, 2015 @02:32PM (#10228747)
> Git isn't so awesome that it's inconceivable that people would be willing to use something else.

Inconceivable? No, I just asked why. Git to me is a great solution for source control, so I want to understand it's deficiency for handling large amounts of code, especially considering it is used for managing Linux source and handles everything in GitHub I've used quite well.

[ Parent | on HN ]
Re:Google Is 2B Lines of Code, All in One Place by spectral321 on Wednesday September 16, 2015 @02:42PM (#10228809)
Git is fine at scaling horizontally (many repositories), so things like GitHub work well. Neither Git nor Mercurial do well at scaling to a single massive repository; getting Mercurial to do so was easier because we can implement it as out-of-core extensions (see http://bitbucket.org/Google/narrowhg, for example).

There's also internal-only things to deal with, such as the CitC integration. Because of some of the design decisions behind CitC, storing a typical .git or a .hg directory in CitC is essentially impossible. Mercurial's .hg directory is intentionally a black-box - you interact with the repo using the hg tool. The .git directory can be seen as an API - there are at least three implementations that matter for our purposes, and if we change something in the .git directory, we lose the editor/IDE integration powered by the ones we didn't fix (or we did fix, but they haven't been released yet; some of those are linked into products that have a commercial release cycle).

[ Parent | on HN ]