It often feels to me like people are using that as a justification of the quality of their hiring practice. What's interesting is hiring practices vary greatly across the board but mostly in aggregate seem to produce this "we only ever got a few duds" result.
My hypothesis has now shifted to "most people can do most jobs". Or to restate it "There are only a few duds. Period. And hiring processes produce mostly random results. That's why we see a few duds come through."
Admit the "no brainers" in the upper right. Reject the "no ways" in the lower left. Then take a harder look at those in the middle. But the reality is that you could probably take an arbitrary subset of the middle group and they'd probably do nearly as well/contribute as much/etc. as the final selection you sweated over.
I've also been on the selection committee for various conferences over the years and that also often ends up working the same way to a significant degree.
There’s people who came off really well in the interview who turned out to be terrible hires. And equally there’s been people who seemed woefully inexperienced but I liked their honesty so took a gamble on them and they turned out to be some of my best hires.
Then there is everyone in between.
You have all of these processes in place to help your practical judgement but at the end of the day it’s really just a lot of luck because it’s trivially easy to game interviews and some really good engineers are often really bad at interviewing. So pulling those who interview well but are rubbish from those who don’t interview well but are brilliant is very difficult at times.
Heck in one interview I got nervous and started rambling on about FreeBSD instead of Linux. I was easily qualified enough for that job but that was a terrible interview on my part and I honestly wouldn’t have hired me.
It's almost like we're trying to solve a taylor series of work efficacy (of an individual on a given team) by capturing a very small number of data points in stressful situations (interviews) and doing a hell of a lot of extrapolation.
I myself have been on interview committees and voted to reject candidates who got hired, and turned out to be good. Why did I want to reject them? Don't really know, just got a bad vibe in the interview more than anything. We didn't do much technical screening at that company. Decisions were mostly based on past experience and references, and whether you seemed reasonably intelligent and sociable in the interview. I think that we hired a real dud maybe once, so it seemed to work.