News for hackers. Stuff that matters.
Re:Why Google Stores Billions of Lines of Code in a Single R... by makecheck on Tuesday June 28, 2016 @09:52AM (#11993766)
There are advantages and disadvantages to consider. Clearly monolithic repositories allow you to leverage commonality but it’s not free.

For example, single-repository environments may require you to check out everything in order to do anything. And frankly, you shouldn’t have to copy The World most of the time. Disk space may be cheap but it is never unlimited. It’s a weird feeling to run out of disk space and have to start deleting enough of your own files to make room for stuff that you know you don’t care about but you “need” anyway. You are also wasting time: Subversion, for instance, could be really slow updating massive trees like this.

There is also a tendency to become too comfortable with commonality, and even over-engineer to the point where nothing can really be used unless it looks like everything else. This may cause Not-Invented-Here, when it feels like almost as much work to integrate an external library as it would be to hack what you need into existing code.

Ultimately, what matters most is that you have some way to keep track of which versions of X, Y and Z were used to build, and any stable method for doing that is fine (e.g. a read-only shared directory structure broken down by version that captures platform and compiler variations).

Re:Why Google Stores Billions of Lines of Code in a Single R... by slededit on Tuesday June 28, 2016 @10:08AM (#11993883)
Companies that do this don't require you to check out the world, that would be craziness. Perforce allows for partial mappings and is used by pretty much all the big companies. Even Microsoft's internal source depot is just a fork of perforce.

Dependencies are usually handled by a binary import/export system where you pull modules from a shared "build the world" system from a known checkpoint.

Re:Why Google Stores Billions of Lines of Code in a Single R... by wocram on Tuesday June 28, 2016 @10:37AM (#11994112)
Both git and hg supports shallow (low history) and sparse (selective files) checkouts as well, facebook has a blog post somewhere about how they built it into hg.
Re:Why Google Stores Billions of Lines of Code in a Single R... by kmiroslav on Tuesday June 28, 2016 @10:47AM (#11994186)
It's still not the same as perforce's mappings. Even with shallow repos, git will force you to download the entire world each time you pull or fetch.
Re:Why Google Stores Billions of Lines of Code in a Single R... by jahewson on Tuesday June 28, 2016 @11:54AM (#11994760)
That's not really the entire world though, because the history before the shallow clone will not be pulled or fetched. You'll get all the new stuff though.
Re:Why Google Stores Billions of Lines of Code in a Single R... by slededit on Tuesday June 28, 2016 @12:17PM (#11994972)
I don't think the scale of these places are fully appreciated. Your local enlistment could be a few hundred gigabytes and that is only the head revision of stuff you work with daily. Keeping more than just the current revision around is a lot of data for even a beefy workstation.
Re:Why Google Stores Billions of Lines of Code in a Single R... by dilap on Tuesday June 28, 2016 @10:10AM (#11993899)
From the article, they have a fancy system that doesn't require you to actually have the whole repo mirrored locally.
Re:Why Google Stores Billions of Lines of Code in a Single R... by oaktowner on Tuesday June 28, 2016 @11:27AM (#11994503)
Citc is, in a word, amazing.