Even if the test is good, it's probably calibrated on WEIRD [0] subjects, so is really only good as another kind of gatekeeper - you can probably use old-school IQ tests as an excuse to reduce diversity. People who aren't WEIRD may not do as well on these tests (even though they're just as "intelligent" (parenthetically bracketing the ill-defined term for a moment)), so you have your reason that you only hire white dudes. But that's a cynical extension of logic there.
Most people, when approaching a psychometric test like one of intelligence or personality make the first incorrect assumption that modern cognitive science can actually form a construct like "personality type" or "intelligence" that's stable across all cultures and norms, testable, and repeatable. That's just not really true, and by and large what makes it to public consumption is pseudo-science.
Diverting just a little bit here, but MBTI [1] is another good example of bad use of psychometrics in business. There's about as much science supporting Myers-Briggs tests as there is astrology, yet people still use MBTI for actual decision making. Types aren't shown to be stable and they're not well-clustered (meaning you can be a mix of introvert and extrovert, or show signs of both depending on the day, context, mood). Zodiac Sign and MBTI Type should only be used as pick-up lines in a bar. "Hey, babe, I'm an INTJ, so I'm not going to say anything else."
There are good uses of psychometrics, but they're rare enough that I take any reliance on any kind of psychometric during hiring as kind of a bad sign.
0: http://blogs.scientificamerican.com/primate-diaries/the-weir... 1: https://www.psychologytoday.com/blog/give-and-take/201309/go...
I do find it surprising that Raven's appears to be more susceptable to the Flynn effect (people keep getting better at IQ tests, for an unknown reason).
Completely agreed on the MBTI, but its surprisingly difficult to convince people that's its useful.
FWIW, I agree with most of what you said above, I was just pointing out a counter-example. (Psychologist/psychometrician here).
Secondly, in this context, the assumption is that not only does this generalized attribute exist in humanity and we can measure it, but this attribute causes success in $job. That's a further assumption that I'd call into question. Sure, plenty of people who do well on $test do well on $job, but those could easily be covariate or epiphenomenological relationships rather than causal ones.
I don't doubt that for some values of $test, there's some discriminatory power predicting the presence of success, especially in more monocultural environments (affluent, often-white, often-male, often-young startups, to caricature). But I do believe that this kind of test would suffer from strong false negatives as many kinds of people who would otherwise be excellent programmers (or whatever other $job) are rejected for not doing well on $test.
That's why I worry about anyone who puts real stock in these kinds of tests - it is (generally) indicative of a kind of science blind spot in the user. People are critical and skeptical of all kinds of other scientific claims, but psychological measures tend to get a pass.
I imagine the fact that a lot of us tend to score well on IQ tests has a lot to contribute here. When I found out how many standard deviations above the mean my IQ score represented, I was pretty excited about how awesome I was. When I realized that the test was probably baloney, I had to figure out more interesting ways to shore up my self-esteem. Personally: I bake bread and fish for compliments on the quality of my sourdough.
I learned, as an adult, that I had taken an IQ test while young and had scored in the 99.97th percentile. I think that I got that not everything benefits from what IQ tests measure. That said, I had already started seeing great success in a knowledge industry. It is clear that you also have great linguistic intelligence. I don't think you could completely dissociate your IQ test results from that intelligence.
The fact that certain groups of people tend to test well (or, taken conversely, that there are so many reasons why otherwise "intelligent" people don't test well) just exacerbates this problem and I think the central claim is still reasonable: things like IQ tests aren't necessarily strong predictors of tech skills, are dangerously close to pseudoscientific when misapplied, and have enough other theoretical problems that they should probably not be utilized during a hiring process.
As to my own linguistic skills, I contend that a lifetime of reading as well as a MS (in computer science), a Ph.D (in cognitive science and education), and ten years afterwards in academics including a professorship has prepared me as a writer. Did I have some initial Potential that gave me a head start? Maybe. If nothing else, I had the head start of my general introvertedness and a love of both reading and geekiness. A short test that purports to measure a fixed potential somehow inherent in someone is going to be pretty flawed.
But my "I don't doubt" phrase was more along the lines of ceding a central point: there probably do exist certain inherent characteristics that vary between people and provide some kind of predilection or head start. I'm just not yet convinced that the scientific community has really identified them yet, or that they're really able to effectively measure them yet. Instead, we get proxies that have a very high false negative rate, especially among otherwise-marginalized groups.
Anyway, four paragraphs is probably enough here.
I had an MBTI test pretty much forced on me a while ago (senior engineer would be unhappy if I didn't take it). I argued very forcefully against it, saying it was like astrology, that it made me feel like the company was asking me to participate in some religious ritual when I was an atheist and so on. I'm not in that company anymore and one major reason I ran was that they allowed pop psychology like that to dictate decisions.
The worst I've seen was where I had to do 100 questions and half were spatial intelligence (gameable and wtf). It looked like an incredibly expensive piece of software. Didn't get the job, thankfully, but the company's now known for having a policy of not being able to talk about work out loud if you're in IT. The other places that gave me tests had dissimilar, but equally insane workplace policies.
Rambling list: -At some level, the person interviewing you (or management) doesn't trust their intuition to find the right candidate.
-It's an easy way to throw someone out of a pool despite all else. There's a reason it's illegal to do IQ tests.
-Mental health issues such as dyslexia or ADD.
-I've taken maybe five intelligence tests for work and none of them have been the same. Where's the standard?
-Companies have a weird tendency to use puzzles such as the "three light bulbs" one. All that does is test if you've heard it before. "I want to see how you think" makes no sense when the question's on paper and there's 100 questions just like it.
-A lot of these tests are gameable since you can just practice beforehand.
-Test anxiety sucks.
Unrelated to that -- one of my personal red flags is poor handling of the post interview follow up. If a company drags their feet, gives conflicting signals about next steps, etc., they probably are a mess.