Most of that team left the org within the next 24 months, and none of the teams they joined got better. Some got worse.
Applying the “promote your best IC to be a manager” mindset applies to teams as well, and it’s a damn shame, because in both cases it usually results in the opposite of what anybody desires.
I think it makes sense to spend some money on this kind of charlatan "rigor" from a legal CYA perspective. But I think the amount of energy and time and resources spent on it far exceeds what is reasonable for CYA, and that money would only be spent the way it is if either/and/or 1) the people making the decisions to do all this extra analysis benefit from it personally in some way, or 2) the people making the decisions truly believe that the data/analysis will yield productive results, or 3) that the people making the decision believe that this much charlatan-rigor is actually necessary for legal "CYA" purposes. I just feel that #2 and #3 reflect such a broad lack of understanding that I'm surprised there are no major corporate leaders bucking the trend.
For hiring, specifically, I believe it's proven in mathematical theory that the best thing you can do is to have a two-stage hiring process:
1) only eliminate/select candidates based on provably strong signals. This will result in a massively large pool of candidates who "pass" the screening for each position.
2) use a random number generator to pick someone from that pool.
A lot of people would propose using the algorithm from the secretary problem[0], but that algorithm assumes that you can "grade" on a candidate's quality on a continuous scale. I don't believe we have the ability to judge a candidate that is "60% likely to be amazing" vs. "80% likely to be amazing", so that algorithm won't work.
We can, however, measure in a binary way whether we were happy or unhappy with a hiring decision, and determine which pre-hiring signals had any actual strong correlation to being happy with the hiring decision. Filter candidates based on that, then pick a candidate at random. Any additional filtering will result in systemically picking worse candidates at scale.
Metrics are a technical solution, and as organizations are social constructs HR is making the classic software engineer mistake and trying to solve a social problem(how do we identify high performers) with a technical solution(generate numbers and use those to identify high performers). The reality is that team performance doesn't yield to analytic techniques because it's so markedly a result of all of the people together and not specific individuals that it's not even funny.
HR defends the company against legal attacks from outsiders and employees alike.
Not all people working in HR have this as a prime motivation, but that's why HR exists.
I like working for my manager, and think he is a very rational and no BS kind of person. However, I have butted heads with the upper management above him at times for what I believe are good and legitimate reasons.
During my last performance review, I had to fill out all the typical questions and surveys that us grunts are forced to answer. While reading/answering some of the questions, I just get this sinking feeling that it's all just a dog and pony show. So, in a moment of distraction while filling out all the questions, I decided to go talk to my manager directly.
I simply asked him to his face, "Is there any answer I can supply to these questions that will change the opinions of upper management about me?"
All my manager said back was a simple, "Nope!"
More to the point, objectivity may not actually matter much because so much of how a team performs is down to how individuals within the team gel, and you can't train someone into having a personality and work style that gels if they don't. Your coworkers can learn to handle criticism but they can't learn to be brash and assertive if it's genuinely not in their personality.