News for hackers. Stuff that matters.
Re:Monorepoize – Bash scripts for creating a monorepo out of... by cryptica on Monday April 27, 2020 @04:43AM (#22994061)
Big corporations are extremely inefficient. Why does everyone want to copy them?

And why do we use all the bulky tools they produce when there are far simpler, better and more open alternatives available?

Re:Monorepoize – Bash scripts for creating a monorepo out of... by beagle3 on Monday April 27, 2020 @04:56AM (#22994133)
Because it turns out big corporations that use multirepos tend to be even less efficient (about versioning) than those that use monorepos.

What are those far simpler, better, more open alternatives?

Re:Monorepoize – Bash scripts for creating a monorepo out of... by hocuspocus on Monday April 27, 2020 @06:29AM (#22994553)
> Because it turns out big corporations that use multirepos tend to be even less efficient (about versioning) than those that use monorepos.

How do you measure that exactly?

Re:Monorepoize – Bash scripts for creating a monorepo out of... by beagle3 on Monday April 27, 2020 @06:46AM (#22994641)
Personally, informal survey of people I know who work at big corps.

Google and Microsoft have both evaluated this internally and reached that conclusion, with some of the evaluation criteria and conclusions publicly documented (though I don’t have links available on my phone, google will likely find them for you)

Re:Monorepoize – Bash scripts for creating a monorepo out of... by hocuspocus on Monday April 27, 2020 @07:08AM (#22994733)
Microsoft doesn't really use a monorepo.

This leaves companies like Google and Facebook that made a decision fairly early in their existence, set constraints, and then spent several hundreds engineer-years into developing their own infrastructure and tools to support that decision.

Does it work? Yes. Is it more efficient than what other companies of similar scale are doing? I don't believe anyone knows.

Re:Monorepoize – Bash scripts for creating a monorepo out of... by beagle3 on Monday April 27, 2020 @08:05AM (#22995014)
From reading articles, I got the impression Windows is one monorepo (but it doesn’t include Office) and e.g. Visual Stduio is another. So it’s true that Microsoft doesn’t use a monorepo as a whole, but e.g. just Windows itself is a codebase significantly larger and more diverse than 99% of companies would have, and it does have multiple “independents” products - Notepad is an independent module from Solitaire and they are both independent of essentially everything else, whereas e.g. File Explorer, the win32 subsystem and the kernel often need to be updated together for new features.

GoogBook made it early, but re-evaluated. For sure, they found the cost of switching is higher than any potential benefit. I must say that I have not found anyone describing benefits other than “git is faster for smaller repositories” (which was very true but is only slightly true with sparse checkouts and shallow branches these days) and “I like it that way”.

Yosefk has a convincing article about “if your culture is bad, it doesn’t matter; if your culture is good, monorepos are technically easier until you hit some scale wall but then they don’t become harder that multirepo at that scale” IIRc.

But w.r.t Microsoft I admit I’m not very well informed, thank you for setting the record straight.

Re:Monorepoize – Bash scripts for creating a monorepo out of... by ex_amazon_sde on Monday April 27, 2020 @06:40AM (#22994608)
> it turns out big corporations that use multirepos tend to be even less efficient (about versioning) than those that use monorepos

Amazon disagrees.

Re:Monorepoize – Bash scripts for creating a monorepo out of... by sokoloff on Monday April 27, 2020 @07:43AM (#22994891)
Does Amazon optimize for efficiency in software engineering, or just for the overall quantity of output? My sense, having spoken with Amazon SWEs and a fair number of engineering leaders is that they optimize for overall output, but not for efficiency (output per worker).
Re:Monorepoize – Bash scripts for creating a monorepo out of... by beagle3 on Monday April 27, 2020 @07:53AM (#22994942)
Well, the measure of efficiency most of the world cares about is “overall profit”, which — around the efficient frontier — has a good proxy with “overall output”.

I feel your “output per worker” measure of average individual efficiency is not a good metric to optimize - just fire everyone except your best employee and you’ve maximized it.

Edit: employer -> employee typo

Re:Monorepoize – Bash scripts for creating a monorepo out of... by sokoloff on Monday April 27, 2020 @07:59AM (#22994982)
I think your first point is good and agree with it. Even within optimizing for that (rather than output/worker or output/salary), I think it's fair to examine whether specific choices are optimized for the same case that you are facing. Not everything done by very successful companies should be replicated in your business.
Re:Monorepoize – Bash scripts for creating a monorepo out of... by unixhero on Monday April 27, 2020 @05:17AM (#22994245)
Proper secure development lifecycle approach for all software projects producing code?
Re:Monorepoize – Bash scripts for creating a monorepo out of... by beagle3 on Monday April 27, 2020 @06:03AM (#22994446)
What are those?

There are tens of such approaches, many of them practiced at large corps, and in my experience almost none of them deliver the supposed benefits, with few exceptions (NASA, MISRA) that are cost prohibitive except for very narrow fields.

And what does that have to do with “copying large corps” and/or monorepo vs multirepo?

Re:Monorepoize – Bash scripts for creating a monorepo out of... by cryptica on Monday April 27, 2020 @05:25AM (#22994273)
I think it's because corporations treat developers like untrustworthy and replaceable cogs in a big machine - IMO, this is what's wrong, not multi-repos. If corporations gave their developers responsibility and ownership over the maintenance of specific repos which are heavily depended upon, then multi-repos would not be a problem. It's a great source of pride for developers to feel responsible for something and to take care of upstream developers.

About simpler alternatives, non-corporate tools and libraries are often much better than corporate-built ones. For example, React was built by Facebook and is inferior to VueJS which was built independently yet React is much more popular.

On top of this, I contend that if Evan You (the creator of VueJS) hadn't worked at Google, very few people would be using VueJS today; but the popularity has more to do with public image than technical merit. VueJS is associated with Google even though it was written independently; something so simple and elegant (especially at the time it came out) could only have been written by an indie dev. Too many cooks do spoil the broth unfortunately.

Re:Monorepoize – Bash scripts for creating a monorepo out of... by beagle3 on Monday April 27, 2020 @06:28AM (#22994550)
I disagree about the nature of the problem.

You can’t give sole ownership of a project people depend on to just one person - a bus factor of 1 is unacceptable in most circumstances. Busses happen, people leave, etc. Some things are also too big for one person.

And when you give it to a team of proud capable developers, you will get ego wars, blame throwing, etc. with probability that increases quickly with team size.

And that’s with capable, proud, responsible, cooperative people.

But not all your people will be like that all the time - some will be burnet our, some have no passion and just try to do the minimum to not get fired. Some are sociopaths who actively try to shift work to others and only pride themselves on work they managed to avoid.

Mom and pop software shops can afford non-replaceable-cogs, Google can’t (and afaik didn’t even in the old days when they were run by engineers and not MBAs).

And this whole discussion of copying corps was in the context of mono Vs multi repo....

(For the record, VueJS and Evan You are awesome)

Re:Monorepoize – Bash scripts for creating a monorepo out of... by ratww on Monday April 27, 2020 @09:34AM (#22995653)
Giving ownership to a single person doesn't imply a bus factor of one. As long as someone can take over, it's not a problem.

I've had this happen lots of times in my career, and most of the time dealing with such projects was faster than onboarding in a company-owned repo by at least an order of magnitude. I really mean it: minutes instead of days to be productive.

In my experience most engineers have the best intentions and want to do the best work possible, until you force them into working in a bad environment where he's treated like... a cog in the machine with zero ability to influence his own productivity, happiness or even the quality of his code.

The biggest reason large companies "can't afford" those things is because they're hell bent on turning engineerings into replaceable cogs, out of ideology.

Re:Monorepoize – Bash scripts for creating a monorepo out of... by beagle3 on Monday April 27, 2020 @10:27AM (#22996117)
> In my experience most engineers have the best intentions and want to do the best work possible

I agree, most of them do - but I've also been in a place that highers the best of the best, and despite the best intentions, some people cannot put their ego aside and the overall result is between less-than-ideal and downright-horrible

Re:Monorepoize – Bash scripts for creating a monorepo out of... by ratww on Monday April 27, 2020 @10:51AM (#22996372)
> ...until you force them into working in a bad environment where he's treated like... a cog in the machine with zero ability to influence his own productivity, happiness or even the quality of his code.
Re:Monorepoize – Bash scripts for creating a monorepo out of... by beagle3 on Monday April 27, 2020 @12:04PM (#22997091)
I disagree (unless your "bad environment" gets tautologically redefined to mean any place where people with good intentions don't get good results.... which you might).

I was working in a place that employed incredibly smart and capable but highly opinionated people, and did not treat them as cogs, and did let them influence their productivity, happiness, and quality of the code.

However, there was a major philosophical disagreement between the "functional programming" camp and the "object oriented" camp (with a minor "procedural programming" camp), and designs occasionally became "I am smarter than you" contests, the overall system being extremely incoherent because every side wss able to influence their own productivity.

It worked, because they were smart. It was not a bad environment for any single person. But it would have been a much better place for everyone if there was a benevolent dictator (but nevertheless, dictator) who enforced one way or the other - the randomly reached middle ground has all of the deficiencies of all approaches, and none of the benefits.

Re:Monorepoize – Bash scripts for creating a monorepo out of... by ratww on Monday April 27, 2020 @01:13PM (#22997750)
If "it worked" and "was not a bad environment", I really don't see the issue here (unless your "it worked" gets redefined to mean "it didn't work"... which you might). Sounds like a nice and intellectually stimulating environment, and that they had best intentions. Honestly, someone wanting a dictator has bigger ego problems than those people.
Re:Monorepoize – Bash scripts for creating a monorepo out of... by beagle3 on Monday April 27, 2020 @02:52PM (#22998733)
Very long onboarding, impedance mismatches everywhere, tons of unneeded glue logic. Lots of consternation. It worked in the sense that it wasn’t a failure like a significant number of software projects. It could have worked significantly better if the people involved didn’t feel they needed to show off.

And my model for A dictator is Linus Thorvald, Guiro van Rossum, Andreas Rumpf; if you can’t appreciate the management model and abilities that a benevolent dictator brings, we will just have to agree to disagree.