News for hackers. Stuff that matters.
Re:Why Google Stores Billions of Lines of Code in a Single R... by twblalock on Saturday January 11, 2020 @03:15PM (#22022000)
Don't try a monorepo unless you have a team of engineers who can work full time on maintaining the build tooling, enforcing good practices, and training the rest of the engineers on how to behave in a monorepo.
Re:Why Google Stores Billions of Lines of Code in a Single R... by vbezhenar on Saturday January 11, 2020 @03:44PM (#22022176)
I work in a tiny team (basically 2 people), but we have to work on multiple small projects (around 200k LoC total) including few shared libraries on Java. We used different repositories before with all releasing rituals, etc. But I tried monorepo and "all use latest" and it worked surprisingly awesome for us. Changes across repositories are much easier to do than before.
Re:Why Google Stores Billions of Lines of Code in a Single R... by twblalock on Saturday January 11, 2020 @11:51PM (#22024562)
A team of two people can work with pretty much anything. The hard part is figuring out how to work with 100 or 1000 software engineers who all commit to the same repo.
Re:Why Google Stores Billions of Lines of Code in a Single R... by forwhomst on Saturday January 11, 2020 @03:35PM (#22022113)
If your organization lacks the ability to enforce good practices, train engineers how to behave, and otherwise acculturate people to your ways then there is no particular style of repo that can save you from failure.
Re:Why Google Stores Billions of Lines of Code in a Single R... by vanniv on Saturday January 11, 2020 @03:49PM (#22022201)
Indeed, maintaining the greenness of the build in a giant monorepo is extremely expensive.

It isn't clear to me that it is worth the investment, even for Google -- though it clearly world's "well enough"

Re:Why Google Stores Billions of Lines of Code in a Single R... by ori_b on Saturday January 11, 2020 @05:30PM (#22022828)
So is maintaining the greenness of a fragmented repo. The only difference is when you find out about the breakages.
Re:Why Google Stores Billions of Lines of Code in a Single R... by cipherzero on Saturday January 11, 2020 @07:43PM (#22023550)
Yes. Thank you! From my experience multi-repo doesn’t make problems go away, it just defers them and introduces others. I’m not against multi-repo or a strong proponent of monorepos... each have their strengths and drawbacks and there is a time and place for either. I struggle to deal with folks who firmly lie in one camp or the other.
Re:Why Google Stores Billions of Lines of Code in a Single R... by vanniv on Sunday January 12, 2020 @06:01PM (#22030296)
Agreed!

Multi-repo makes integrations harder, because you find out about breakages later -- which means that it might be harder to identify the cause and more likely that someone depends on the behavior that broke you by the time you notice it.

But, multi-repo makes local development faster and cheaper -- you're insulated from the churn of everybody else's check-ins. You don't have to constantly refactor everything every time some dependency makes a minor tweak.

You get more done and have a smoother development lifecycle -- but you're going to keep falling behind your dependencies unless you invest in keeping up -- and that part of the process is more unfun the less often you do it.

I currently live in a monorepo. I don't love it. But I've also not loved the multi-repos I've lived in, so... shrug

Re:Why Google Stores Billions of Lines of Code in a Single R... by m0zg on Saturday January 11, 2020 @04:46PM (#22022553)
It's only expensive if you don't have tests and CI. If you do have tests and CI, it's much cheaper to maintain one repo than three dozen smaller ones: garbage simply can't get into master. Note also, that if you want to ship high quality software, having a fragmented repo doesn't free you from the need to have tests and CI, if anything it leads to duplication of effort.
Re:Why Google Stores Billions of Lines of Code in a Single R... by vanniv on Sunday January 12, 2020 @06:09PM (#22030351)
I currently live in a monorepo that has pretty good CI and test coverage. That means that I know every time I have work to do because something changed (because the tests go red). That's great. But I still have to do the work every time someone changes anything that affects me
Re:Why Google Stores Billions of Lines of Code in a Single R... by m0zg on Sunday January 12, 2020 @06:14PM (#22030390)
You're doing it wrong: if _they_ want to change something that affects you, _they_ should be the ones doing the work to not break you. The situations where _you_ have to do anything more than a code review of such changes should be very rare. That's how it works at Google.

And besides, it's not like this gets any easier if you have multiple repos. Eventually you have to roll forward your dependencies, and doing the work that goes along with that.

Re:Why Google Stores Billions of Lines of Code in a Single R... by vanniv on Sunday January 12, 2020 @06:37PM (#22030541)
That's only how it works at Google if the "they" is sufficiently organizationally close that you actually care if you break them (and you'll only even know about it if you're close enough that you run their presubmit)
Re:Why Google Stores Billions of Lines of Code in a Single R... by m0zg on Monday January 13, 2020 @03:20AM (#22032751)
No, you don't have to "care". Presubmit _won't let you_ break them. That's the whole point of presubmit - to make sure tip of tree is green 100% of the time. You can't submit something that breaks the build _or_ breaks the tests. You can work with the code owner to make sure you can commit in the first place, but you don't get to just push your changes through and have the code owner deal with the fallout. Facepalm.