If you grossly simplify it to its innermost core, making development scalable means that if you have 10 times more people you can have 10 times more features/bugfixes/speed improvements/... The only way to do that is to make sure that an additional developer doesn't depend on other developers to work, and that can only happen if everything is properly documented, the build instructions are up-to-date, the processes are clear, basically anyone can start from scratch and get up to speed without the help of anyone else.
That kind of organization traditionally doesn't happen in on-site companies, where newcomers are followed by senior people, they have to follow some introduction course to familiarize themselves with the processes, they have to ask many questions every day, they need to be synchronized with other persons which brings some inefficiency beacuse everyone works at their own pace, etc... This all disappears when everything is properly documentend and every contributor can work in the middle of the night if they wish. I think the Gitlab Handbook goes over this quite well and describes a framework to implement that kind of organization but the rules are retrospectively obvious for people already used to open-source (https://about.gitlab.com/company/culture/all-remote/guide/):
- write down everything
- discussion should happen asynchronously. Any synchronous discussion (by text or call) should be only very small points. Whatever the type, write down the conclusions of that discussions
- Everything is public (to the organization), including decisions taken, issues, processes
Another very relevant factor is that people can just clone stuff, create new projects, and everything moves independently. So the open source development model has a pile of solutions for dependency management that team based development doesn't adopt.
But the one thing I don't get is why team based development doesn't adopt those solution. They are not expensive. Yet, even when I was able to dictate into everybody's requirements, I wasn't able to force teams into adopting them. Instead they insisted on synchronizing themselves by much more expensive and less reliable means. My guess is that most developers never dug any deep into open source and have no idea how it's done.
“open source” isn’t a development methodology, or even a distinct set of methodologies.
> The project structure, tooling and documentation that lets new contributors jump in quickly makes software development easy to scale.
Plenty of open source projects don’t have that, nor is there anything restricting those things to open source projects.
It is true that some open source projects, because they see the value in new developers jumping in quickly, prioritize having structure, documentation, and tooling that supports that. It’s also true that some proprietary software projects do, too, because the project owner sees value in that.
There are books written about different authors idealizations of how to do open source development. The fact that many people have written about different methodologies for approaching a particular challenge doesn’t make the challenge a methodology.
And, yes, I am aware that the very approach known as “the bazaar” (from the essay “The Cathedral and the Bazaar”) is sometimes referred to erroneously as “open source development”, which is a particularly glaring error since all of the examples of both “cathedral” and “bazaar” development were open source projects.
Yet every large project behaves in a similar way, with only a few larger variations.
Some companies such as Microsoft, Google, Amazon would disagree.
https://devblogs.microsoft.com/oldnewthing/20180326-00/?p=98...
Bonus chatter: In a discussion of Windows source control, one person argued that git works great on large projects and gave an example of a large git repo: “For example, the linux kernel repository at roughly eight years old is 800–900MB in size, has about 45,000 files, and is considered to have heavy churn, with 400,000 commits.”
I found that adorable. You have 45,000 files. Yeah, call me when your repo starts to get big. The Windows repo has over three million files.
Four hundred thousand commits over eight years averages to around [130] commits per day. This is heavy churn? That’s so cute.
You know what we call a day with [130] commits? “Catastrophic network outage.”
Also, even if we normalized both cases for 'code files only' and/or 'kernel code only', there could still be architectural, code style, and development process differences that lead to different metrics for each project.
If you took the 100 biggest code bases in the world. I would be surprised if more than 10% of them were open source.
[1] https://www.informationisbeautiful.net/visualizations/millio...
And again, that's for a middle size company you haven't even heard of. FAANG, Microsoft, Oracle, IBM for sure will have stuff dwarfing that.
An explosion on the number of lines of code is one of the way development teams fail.
The truth is that Oracle is ridden with coherence bugs, and have a much worse performance picture out of the box than Postgres. But while improving the Postgres performance requires deep digging into the DBMS itself, Oracle has a lot you can optimize just on the SQL interface. But there are plenty of cases where Oracle just can not become as fast as out of the box Postgres, and many where it could in theory, but a bug prevents you from doing things right.
Overall, three is no definitive ordering on the speed of the "good 3" DBMS. It always depends on what you are doing.
Most open-source projects have one to maybe five real contributors, outside the drive-by pull requests to fix some bug.
1. The project structure, tooling and documentation that lets new contributors jump in quickly makes software development easy to scale. In Coasian terms, the transaction costs around development are minimized.
2. It enforces hard interfaces and it clearly separates development from operations. Lack of discipline around these issues is a source of much accidental complexity.
[1] I can’t seem to find the quote, I read it in an interview a few years ago and stuck with me since.