But if you have to fill 5 slots and you have 10 candidates who all scored above 70, you now have to choose between them somehow. And the candidates who scored 95 are legitimately expected to perform the job better than the ones who scored 75, even though the ones who scored 75 would have been better than an unfilled position.
1. there is a test that is a decent proxy for job performance
2. the relationship between job performance and test score above some passing score is linear
These both sound "common sense", but I suspect fail for a huge number of real world scenarios.
The second assumption is not required. If people who score a 95 are only 5% better at the job than people who score a 70, all else equal you'd still pick the person who scored a 95 given the choice.
As for the article, it's not given me particular solid vibes, a feeling not helped by some of the comments here (both pro and con).
Satisfying the first assumption means "still monotonic".
Also, if you had a better test then you'd use it, but at some point you have 10 candidates and 5 slots and have to use something to choose, so you use the closest approximation available until you can come up with a better one.
Sorry, but I just don't agree. There are "qualifying tests" for jobs that I've done that just do not have any sort of monotonic relationship with job performance. I'm a firefighter (volunteer) - to become operational you need to be certified as either FF I or FF II, but neither of those provide anything more than a "yes, this person can learn the basic stuff required to do this". The question of how good a firefighter someone will be is almost orthogonal to their performance on the certification exams. Someone who gets 95% on their IFSAC FF II exam is in no way predicted to be a better firefighter than someone who got 78%.
If there's a test used as the basis of consideration, and some process has decided that any score over X makes the candidate qualified, but then you are later going to claim that actually, given that there were candidates with a score of X+Y, a score of just X does not really constitute "qualified" and the higher scoring candidates should have been chosen, then the whole nature of the test and the ranking becomes rather suspect.
So either everyone who is judged to be qualified really is qualified, and it makes no difference that they were not necessarily the highest scoring candidates ... or ... the test for "qualified" is not suitable for purpose.