Poor management can contribute significantly to OpEx. Compared to ICs, managers have a disproportionate ability to destroy value. Of course, even that is not unlimited. It is true that companies often succeed in spite of managerial issues, but it is only to a point. Once certain lines are crossed from the legal or HR side, the potential collateral damage compared to the average value add can be massive.
With that said, I don't disagree with you that in large enough companies, managers have to do a lot to compromise the success of a company. In practice, there is often enough vertical pressure from and sideways pressures from competing fiefdoms to prevent a single rogue manager from going hog wild. And from the other end, so long as managers are adding (or able to sell the perception of adding) just enough efficiency to the overall organization to support their paycheck, it is true that the only loss is of sanity to ICs, who necessarily must become fungible to a company of that size.
In a large enough company, managers have to do a lot to compromise the success of a company. I'm not saying managers are completely unimportant, but they're not as important as you think.