If you want a better analogy that doesn't belittle senior leadership, try Go instead of Checkers.
One of the keys to successful management and leadership is understanding sustainability. Incremental changes over a period of time beat a huge lift followed by burnout and stasis.
Sustainability is the domain of entrenched/ossified/bureaucratic middle managers, moderating or outright sabotaging whatever BS senior leadership is excited about this month.
I think you are severely discounting the skills it takes to lead people, and you are also making an extremely optimistic assumption that companies in the future somehow would be magically more meritocratic than existing companies, when there's no evidence of the sort. On the contrary, people with great interpersonal skills will continue to advance off of the labour of the people with only great technical skill.
Your glorious engineer's revolution will probably not come to pass if history is any indication.
Somehow, the managerial class and the investor class insert themselves between the fruits of the labour and the engineers performing said labour. And if you think that's suddenly going to stop because the nature of the labour changes, then I think you're mistaken.
Enterprise structures rewards rent-seekers, because they are built by, and controlled by rent-seekers, utilizing disposable labour. The only way a power structure like that can be flipped, is through revolution. Or by forming your own company, out-competing these enterprises.
Every QA person I've worked with that is still in the industry has moved on to almost pure test automation. A few years ago that was more like 30% of the role. Now I rarely do almost any manually testing, except for exploratory purposes and to design the test plan. Also for certain things that don't automate well. These days I do more pure development than I have at any other point in my career. It's been super fun, but still quite a change. I've seen colleagues that were more the business analyst flavor of QA get pushed out of the industry when they couldn't/wouldn't adapt.
> If leading people is so goddamn easy compared to software engineering, why do so few engineers want to move into people leadership,
a) because they wouldn't enjoy it as much as their current job?
b) because for engineers, being a manager does not give them status amongst their peer group.
> and why do so many of them absolutely suck at it?
a) citation needed. Is it provable that engineers are particularly worse at leadership than the rest of the population per capita?
b) if engineers tend to suck at being managers, it doesn't show that managerial roles are harder than engineering ones. How good are managers at attempting engineering roles, for comparison?
I am so reminded of Brave New World...
"Alpha children wear grey. They work much harder than we do, because they're so frightfully clever. I'm awfully glad I'm a Beta, because I don't work so hard. And then we are much better than the Gammas and Deltas. Gammas are stupid. They all wear green, and Delta children wear khaki...I'm so glad I'm a Beta"
One additional bullet point I would add: Hiring.
If an isolated HR unit controls hiring and freezes the hiring managers and team out of the process, or effectively alienates them, you're wasting a huge amount of energy.
This is part of the methodology I try to weave into my organizations:
- A thoughtful and literate job posting, purged of gratuitous jargon, will accurately describe the job and foreshadow the company culture.
- Candidates will be evaluated using a simple quantitative assessment of core competencies (see Ch. 21 of Kahneman’s Thinking Fast and Slow).
- Final decision will be a collective decision of the hiring team.
- After hiring cycle is complete, hiring team will hold a retrospective.
- We will acknowledge mistakes and work to correct them quickly and humanely.
I've tried implementing that with a coding take-home task, with "mixed results".
On-paper great candidates often refused.
On-paper good candidates made mistakes because they completed it very quickly, while mediocre candidates invested x10 the time and produced "better objective result".
Comparing code turned out to be much less straightforward or objective than I've anticipated: X handed great consistent well-commented code, but didn't null check inputs, Y covered all cases but left internal class implementation details public, and so on and so forth.
I posit it's still the least bad way to evaluate a candidate. If a 'mediocre' candidate is able to complete the task well, they have still completed the task well.
And, IME, 'great' candidates, even with a storied resumes, can simply be good at self-advancement and job application as much as actual competent at their job.
I don't care how someone looks on paper. Nor should you. Your process should optimize for people who are good at the job. You can look at all the negatives and try and guess "maybe that was a false negative!"...but who cares? What's your rate of true/false positives? That's the only data point you have; optimize for that.
I go so far as to say I think Hanlon needs a new razor. Never attribute to stupidity that which can be adequately explained by apathy.
1. I'm playing chess; senior management is playing checkers. No, not because I'm some sort of genius and they are simpletons, but rather they talk about interchangeable "resource units" and I know these FTE's as "Tom" and "Amy", each with specific strenghts and weaker areas. Big strategic plans cannot differentiate but we front-line managers need to figure out how to position and leverage individuals.
2. Actually, genuinely, caring about your direct reports goes a very long way towards solving a lot of culture problems but...
3. Dev Managers need to put a lot of effort out into their teams to drive change and it is often (most of the time?) not returned. Your biggest enemy is not active sabotage, it's apathy at all levels of the organization. I started pushing for some specific cultural-ish changes about this time last year and, while I was given passive approval & support and lots of latitude to make change, it has been physically exhausting. If I was in an organization that didn't even give me this much freedom it would have been exhausting and pointless.
4. A Dev Manager can make localized changes when not all teams want or can reciprocate, but I'm not convinced yet that the requisite firewalls you need to erect aren't ultimately harmful in the long run. It is a very delicate balance.
In the end my take-away is you need space and time to make any meaningful change, and even that is limited by the crushing inertia of the organization. For me personally it has been physically consuming and I have 6 month & 1 year goals, plus an overall 3-year plan that I'll either complete in the coming year or look for an new opportunity elsewhere.