Convince me that this duplication of work is worse when you have to shoehorn solution X into solving problem theta poorly, because someone 2 degrees (or more) removed from the problem thought that problem iota (which X actually solves) was "basically the same thing" as problem theta.
But this example I don't completely agree with:
> There was no one in a position to see that Team A and Team D were both solving problem theta, and that Team D's solution was better, and Team A should just use Team D's (or, more often, that we needed to detail an engineer to solve problem theta fully and get both teams to use that full solution rather than the jury-rig that each team had built)
I've seen it many times that forcing teams to use the same solution doesn't work as imagined. Often times, teams are solving similar problems. But similarity isn't grounds enough to merge a solution imo. A programming analogy is DRY. Everyone has seen what happens when you take DRY too far.
Some level of management and bureaucracy is necessary. But I think nearly every single company that's ever existed, takes this too far. There are far fewer examples of companies that under manage. I'd argue that all else being equal, under management has less downsides than over management. The reason we don't see companies with too few managers is because of individual interests.
Basically, imagine that a company has the perfectly correct number of managers (contrived example for sake of argument). Then imagine that you can choose to either add or a subtract exactly 1 manager. I am willing to bet that in the vast majority of cases, subtracting the manager leaves the company better off than adding one.
Internal communications like that are a scale problem: in a small company, the matrix of personal relationships is basically full, but as companies get larger, the matrix gets sparser and sparser. By the time you have a 100,000 employees in time zones all over the world your matrix is almost all 0's with only occasional 1s. And so information will travel quite poorly without people whose explicit tasks are to convey information to the right people. That's what managers and internal bureaucracy are supposed to do. Sometimes that information is "we need to build this and not that," sometimes it's "employee 24601 is great and should be given more responsibility," sometimes it's "this project is a Kafkaesque nightmare of un-ending pain that will never be delivered as scoped." But identifying this information and sharing it with the correct other people- that's what middle-management is supposed to do.
Believe me, I am not generally a fan of bureaucracy (as I type this I'm supposed to be fixing a problem that is somewhere in the interface between my 100k employee company and another 100k+ employee company, and it's goddamn terrible), but it does have value beyond just fief-building.