I'm sure people advocating for them have brought up some benefits. But in case they haven't, here's an example I recently encountered at work: for an application we are developing, we needed functionality X. We knew that we'd need X in other projects, too, so we made it a library. I published it to our internal package registry and referenced it in the project. So far, so good. Turns out, feature X wasn't as well understood as we thought it was and we are still changing things around all the time. That meant: * changing the library * cutting a new release and publishing it * updating dependencies in the project and using the new feature
Since this was happening much too often for what was often a one or two line change, I moved the library into the project repo and build it as part of the project pipeline. This removes so much friction around these changes, I find it makes a significant difference. Eventually, I'll move the library back into its own repository, but for now the monorepo is the right choice in my eyes.
This process requires that you are able to work on rewrites and refactorings without having to beg. But it's truly the best way to get from an idea of a library to an actually well designed library.
In some situations it might be you already know what you want and can separate it out from the beginning, but if you are coming up with anything remotely novel, then you want to follow this process.
A monorepo will still contain libraries like these. It will also contain vendored code (although monorepos help reduce the amount of duplicate vendored bits of code). Making such changes will be easier in a monorepo once your abstraction starts to solidify. But the process you should go through should still be similar. The reason for not having two things depend on your wobbly abstraction too early is simply that it will inevitability lead to you prematurely solidifying uncertain abstractions.
This is the mistake: you should make it a library when you do need the functionality, not when you "know" you "would need" it.
Now I will agree that developers are often wrong. We often think something we write is important than it really is and so make it reusable when it never will be. We often think something won't be reused when it really would be. This is a hard problem, but that doesn't mean you should not think about it and work hard to get things right.
In your particular case, if your library becomes too popular, your one or two line implementation detail changes ripple out and trigger rebuilds of too many downstreams, many of which will have flakey tests and fail your MR. If most users are not actually depending on that functionality, or if you simply are doing semver properly, then you could avoid rebuilding those dependencies. Eventually the builds take too much time and CI rejects the pipelines or you continuously bump the build timeout but wait longer and longer for your changes to go live. You can solve these problems, if you invest in more monorepo tooling.
Similarly, once you are too popular a library in a monorepo, you will never do any atomic breaking API changes since it would require updating too many downstreams. Instead you will fake version it: add a new API, migrate users to it in multiple commits, delete the old version. Some of these migrations run out of steam midway in the biggest phase: phase 2. This approach does have the benefit of forcing the upstream author to make the two versions of the API co-exist.
Of course I am talking about scales where real limits start to break down. When your codebase is larger than your ram, an index won't fit into RAM anymore and every code search requires disk or network I/O. Eventually your repository doesn't fit on disk anymore and you interact with it with a special IDE and only download things you start editing or perform sparse checkouts in the first place so discoverability is again a problem.
Edit: of course some problems crop up sooner than hard limits are reached, like the flakey test issue I mentioned as well as visibility and control of changes to actual maintainers.
And how is sparse checkout worse for discoverability? With multiple repos it's even harder to find what you want sometimes if you are talking about 100's of random repos that aren't organized well.
In abstract I agree. However when I'm trying to get my code working having test failures in code that isn't even related to the problem I'm working on is annoying and I can't switch tasks to work on this new failure when the current code isn't working either.
First, the one where everyone does everything correctly: CI executions do not run serially because when too many people are producing a lot of code, you need them to run at the same time. So you have two merge requests which are done around the same time A and B, they each see a commit C before each other. Say merge request A deletes a function or class or whatever that merge request B uses. Of course merge request A deleted all uses of that function but could not delete the use by B since it was not seen. A + C passes all CI checks and merges. B + C passes all CI checks and merges. A + B + C won't compile since B is using a function deleted by A. If you are lucky, they touch the same files and B doesn't merge due to a merge conflict and the rebase picks it up, otherwise broken master.
Then you will typically have emergency commits to hotfix issues which might break other things.
Then you will have hidden runtime dependencies that won't trigger retests before merge due to being hidden, but every subsequent change to that repo will fail.
Then you will have certificates, dependencies on external systems that go away.
This happens no matter which repo type. Even worse if a project chooses to update after a while, its far more painful having to do the changes after losing the context you had when you did the original changes.
If you want a monorepo, libraries being on the same version is a feature, and it keeps you from diverging.
But this isn't a problem. If 1.0.0 is a finished product then why do you ever need to move to 2.0.0 if you don't need the new features?
The issue in the monorepo is that if you are too popular the change must happen all at once or with copying (fake versioning, like people who version excel files by suffixing with dates), which places pressure on maintainers to not fix design mistakes.
It isn't a feature of the monorepo because you can still diverge by copying, forking or merely stopping support for the old library and this becomes more and more necessary at scale: you lose the feature you thought you wanted the monorepo for.
Fair point, I assume (from personal experience at the places I've worked at) that updating the library is inevitable and doing so at a later date tends to be more painful than doing these migrations all at one.
> It isn't a feature of the monorepo because you can still diverge by copying, forking or merely stopping support for the old library and this becomes more and more necessary at scale
This is a problem if all the projects in the monorepo are not actually related. But imagine if all these subprojects are bundled as one OS image, in that case it is very rare that you want multiple library version.
At very large scale I can see your point, I don't have experience there so I can't really argue.
When I read the white papers by google or uber on their monorepos, when I see what my company is building, it is just a custom VCS. Everything that was thrown away initially gets rebuilt over time. A way to identify subprojects/subrepositories. A way to check out or index a limited number of subprojects/subrepositories. A way to define ownership over that subproject/subrepository. A way for that subproject/subrepository to define its own CI. A way to only build and deploy a subproject/subrepository. Custom build systems. Custom IDEs.
The entirety of code on the planet is a polyrepo and we don't have problems dealing with that scale like we would have if we stuffed it all in one repo like this debian monorepo shows. Independence of lifecycle is important, and as a monorepo scales up people rediscover that importance bit by bit.
If you publish something as a library for someone else, you should generally commit to supporting the public interface of that library as if it were a long-term support release. Exactly how long that should be depends on many factors, of course. But as a rule of thumb, I would recommend supporting this library version for at least two years. If that doesn't make sense, don't release it or look for a subset, where it makes sense.
(but also keep in mind this is an internal library for likely a couple of internal apps, supporting BC for 2 years is in most cases unwarranted)
The point is you have some code that you want to reuse, and either within a monorepo or between multiple repositories.
What the parent actually did when copying the library/module code was creating a sort of a a long-term support version for himself maintained by himself. His cost-benefit analysis told him that this is better than always trying to keep up with the changes of the library/modul.
My comment was aimed at setting the bar rather high for when to share reusable libraries. And even for my own modules I rather prefer copying code instead of reusing it in different contexts. There is a reason why the contexts differ and it is typically easier and more economic to adress this differences in its particular place instead of preparing for them in shared code.
The more up-stream a library/module is, the more general it must be, but the less liable to change it should be.
If you drop a library for internal usage only and you want to change the contract, and the tests, library, and api are all in the same repo, you just change them in a single PR and that's done. This works because the single commit hash contains all information. It requires people put their code in the monorepo and hookup their tests, but assuming they do, you can build a reasonable degree of confidence on a green build.
Once you separate the library different repo, and let people consume that as something in their own repos, you need to do the versioning dance as you have no idea if their still is still working.
A lot of people go for the latter because it contextually allows them to ignore the rest of the stack, and there are some pros to doing this, but testing, deploying, versioning, etc, all become more difficult, and that's something struggle with.
Thus, unless you have a crap load of code / commits, monorepos arguably have more advantages than disadvantages.
The reality is, at some point, having repos that are hundreds of GB with hundreds of active PRs also has its own downsides that requires tooling and workflows to combat. Meanwhile, splitting it all up introduces integration problems.
It's definitely pick your poison, though specific requirements and circumstances make specific paths more or less potent.
I have never encounter this use case, tho
The famous 5 minute project.
Happens in coding, car repair, and interstellar travel.
Obviously you want to switch back to a release model once the feature stabilizes, to have all the advantages your internal package registry brings you.
I love being able to easily grep through all the packages source code, and there's regularly PRs that harmonizes conventions across many packages.
Nixpkgs doesn't include the packaged software source code, so it's a lot more practical than what Debian is doing.
Creating a whole distribution often requires changes synchronized across many packages, so it really makes things simpler.
https://github.com/NixOS/nixpkgs
I think it's important to add that they both have the biggest number of packaged software and the most up-to-date software of all distributions even though they have far fewer maintainers than Debian.
> even though they have far fewer maintainers than Debian.
Debian has ~1.5k contributors, nixpkgs lists >3.5k maintainers. (Although that list is not pruned over time)
I think we can say it's a monorepo of packages in this context. Not everything from the Nix ecosystem is there. It could also bundle the website, wiki, doc etc but I don't think it matter too much.
> Debian has ~1.5k contributors, nixpkgs lists >3.5k maintainers. (Although that list is not pruned over time)
Thanks for the info, I heard that a long time ago and never checked myself! It's probably less as you say but still probably bigger than Debian.
I guess it makes sense because it's so much easier to contribute there than to Debian.
Only because of https://www.reddit.com/r/NixOS/comments/zp95a2/comment/j5ko9...
I also had plenty of issues with either packages not being available or not building, last I tried. At least with the AUR, it's generally only the latter you have to worry about ;)
Why spend manual time when it can be automated?
This is the key difference. OpenWRT is a “monorepo” too, but it downloads tar balls etc of the upstream software. Makes sense.
Putting the source code of the upstream software in a monorepo sounds like a nightmare…
Why? If all Your work and knowledge is confined in one repo, there still is a way to work only on that part. Or at least there should be.
But if You want/need to have access to other parts, it's much easier to use monorepo then to search trough 100 of github repos that are somehow dependent but it's not obvious in what way.
The monorepo at Google with the massively distributed build system and all other tooling make it very effective to work with.
You cannot just take one part of that system and expect to magically reap the benefits.
The main benefit I see in practice is that you can immediately see which parts of the system are broken by your change immediately as you test your change (as opposed to waiting when integration happen later when dependencies between modules are bumped in a multi-repo)
Yeah, specifically people only focus on the "repo" bit. Build system, PRs, history browsing, etc all get handwaved away after you stick all the code in a single repo. These are the extra hard parts though!
I think if you were to properly implement "monorepos" in git world it would actually look a lot like a Github _organisation_, rather than a Github repository. Each git "repo" in the "org" would be a workspace with it's own segregated UI so that you can for example check out just that workspace, or see only that workspaces commits or issues -- but some features such as github actions workflow triggers and PRs would be able to span multiple workspaces.
Github doesn't really seem to have put much thought into really supporting monorepos though. They have added some support (e.g. codeowners) to support "repo"-as-monorepo, but have also added some features that could go towards supporting "org"-as-monorepo (e.g. dependabot dependency updates and the dependency graph). But it all stops well short of a complete monorepo toolset.
That said, you can definitely make it work with Git & Bazel. Beats submodules by a mile.
Also another thing: often people end up with the worst of both worlds: one (or two!) big monorepos plus a swarm of many little repos. Now you only pay the price of the monorepo but rarely ever notice the benefits
Quite the opposite, IMHO. Juggling dozens of PRs across many repos for a simple change is the definition of a nightmare.
The flow should be: (1) update a library and release it, preferring backwards compatible changes or a new major version; (2) code that wants/needs the new functionality updates the usage at its leisure. If you are having to keep multiple repos "in sync" with each other, yes, you picked the wrong abstraction. If the downstream repo(s) need to always be on the latest and greatest version of the library, the library code should be vendored with the calling code.
That's how you end up with infinite support costs in a large org. I think the point is that these changes are done atomically.
The alternative is you do the work for them - which you may or may not have the expertise for, and the other team may not even have time for the needed code review.
"It depends" is thrown around a lot; what I've seen working for large companies and on large projects (mostly AAA games, which do use a monorepo typically) is that having a single repo means your dependencies are more likely to be vendored, and updating the vendoring re-runs all tests across all services because dependencies are mapped across the whole project.
Incremental build is also something that becomes more possible, rather than constantly building all source from scratch as we tend to do for smaller projects.
There's a lot to be said for atomic commits, no "merge trains"[0] and things like a "common library" (that everyone ends up building in a large enough team) are just includes and not entire dependencies with all that entails (versioning, updating dependencies etc;).
Having everything in one place also tends to force people to put documentation near their code instead of something else like confluence.
It's pros and cons, and the pro's are kind of meek-sounding until you've experienced it.
[0]: https://docs.gitlab.com/ee/ci/pipelines/merge_trains.html
What do you find nightmarish?
* you can pretend you don't need a proper sandboxing build system like Bazel, Buck etc. and stick to Make/CMake etc.
* you don't need to learn how to use sparse checkouts
It basically lets you avoid learning how to solve the problems you'll inevitably run into for a little while. Not a great reason, but it is a reason.
the truth is, having a large code base is just hard no matter which way you handle it. you'll end up with custom repo tooling for the monorepo or blown up CI/CD infrastructure for many small repos either way. complexity will be conserved; it can be transferred, but can't be removed.
The reason that this is a bad April first joke is that the idea actually has merit. Google runs their company like this. The unthinkable bit is not doing this technically but Debian rearchitecting the way it works. Doing this with Git is a separate topic. It's probably not great for this. But mono repos at this scale are not impossible.
In any event, as has been alluded to in this thread already, any complaint about microservices vs monorepo and switching between them is almost always a tooling complaint. Companies will invest in one of these “philosophies” around code organization then won’t take the time to build the appropriate tooling to support them and then three years down the road some enterprising architect will switch from one to the other and the cycle begins anew
It allows to solve "problems of scale" with technical solutions rather than process solutions.
For example: Let's say there's application app-A and app-B, and they decide they want a library lib-1 that they can share.
If they are in separate repositories, this means multiple pull-requests, it means separate pipelines where the pipeline of lib-1 likely won't include the tests of the applications, it means there will be pull-requests to the library which won't immediately be integrated into the applications so that some poor sod has to take care of breaking changes down the road, etc.
If they are in a monorepo, each application can set up tests that need to be fine with the library-code "as is", so any change to the library needs to work against existing application-tests. The price one pays however are pipelines that perform well - nobody wants to wait hours to get pipeline-feedback and such. A monorepo also allows ambitious folks to shine - it's easy to touch many things at once, or to touch things used by everyone, on a pull-request with high visibility, which is an easy platform to get "street creds" as an ambitious engineer.
A monorepo requires org changes. It requires staffing teams to handle processes and tooling for everyone else. (And it may require something else than git… but git will work OK for reasonably large ones anyway.)
I wrote about some of this here: https://jmmv.dev/2023/08/costs-exposed-monorepo-multirepo.ht...
Boiling frogs and all that.
You'd also be surprised just how little work this actually is if you get it right, versus having to fix 50 different hacked up solutions across your company. It does put strain onto some tooling, but these days git seems to be fine well beyond kernel-scale - when you are this size you have "made it" anyway.
backend, frontend and two shell wrappers for the frontend (electron for windows, cordova for ipad)
The total code base is still small enough for network/file system to not care (~10 devs working 10 years.)
Since we're on Azure Devops, we currently have to create a pull requests for every project. If you change the naming of an API variable, you'll have to create two pull requests. You'll have to review those individually. The automatically triggered test runs will fail against the respective "old" version of each other and create noise.
If, instead, we'd have a structure like this, the changes could be together: /src - backend (C#) - frontend (TypeScript) - ...
Note that we're not planning to have a "shared" library or "common" editorconfig files, or changing _anything_ about code internals. Just tracking the folders in git together.
So many of the supposed problems with monorepos (other than those of pure scaling) seem to basically boil down to "Dr Dr it hurts when I do this"
For example:
"What happens if two teams depend on different versions of the same internal library" - sounds like a reasonable concern no? No! This situation is fundamentally a disaster. On many levels.