They're great for determining which employees will be productive
They're entirely terrible for figuring out which employees will be decent people to spend 9-5, 5 days a week with.
Realistically you need a bit of both
Work samples are my #1 competence filter. They've also formed the basis of great interviews.
As for your second point, sometimes you want to hire a change agent. It takes skill and practice to distinguish toxic personalities from the merely disruptive. Again, standard questions designed from a perspective of empathy and experience have helped me.
Unrelated to that -- one of my personal red flags is poor handling of the post interview follow up. If a company drags their feet, gives conflicting signals about next steps, etc., they probably are a mess.
Even if the test is good, it's probably calibrated on WEIRD [0] subjects, so is really only good as another kind of gatekeeper - you can probably use old-school IQ tests as an excuse to reduce diversity. People who aren't WEIRD may not do as well on these tests (even though they're just as "intelligent" (parenthetically bracketing the ill-defined term for a moment)), so you have your reason that you only hire white dudes. But that's a cynical extension of logic there.
Most people, when approaching a psychometric test like one of intelligence or personality make the first incorrect assumption that modern cognitive science can actually form a construct like "personality type" or "intelligence" that's stable across all cultures and norms, testable, and repeatable. That's just not really true, and by and large what makes it to public consumption is pseudo-science.
Diverting just a little bit here, but MBTI [1] is another good example of bad use of psychometrics in business. There's about as much science supporting Myers-Briggs tests as there is astrology, yet people still use MBTI for actual decision making. Types aren't shown to be stable and they're not well-clustered (meaning you can be a mix of introvert and extrovert, or show signs of both depending on the day, context, mood). Zodiac Sign and MBTI Type should only be used as pick-up lines in a bar. "Hey, babe, I'm an INTJ, so I'm not going to say anything else."
There are good uses of psychometrics, but they're rare enough that I take any reliance on any kind of psychometric during hiring as kind of a bad sign.
0: http://blogs.scientificamerican.com/primate-diaries/the-weir... 1: https://www.psychologytoday.com/blog/give-and-take/201309/go...
I do find it surprising that Raven's appears to be more susceptable to the Flynn effect (people keep getting better at IQ tests, for an unknown reason).
Completely agreed on the MBTI, but its surprisingly difficult to convince people that's its useful.
FWIW, I agree with most of what you said above, I was just pointing out a counter-example. (Psychologist/psychometrician here).
Secondly, in this context, the assumption is that not only does this generalized attribute exist in humanity and we can measure it, but this attribute causes success in $job. That's a further assumption that I'd call into question. Sure, plenty of people who do well on $test do well on $job, but those could easily be covariate or epiphenomenological relationships rather than causal ones.
I don't doubt that for some values of $test, there's some discriminatory power predicting the presence of success, especially in more monocultural environments (affluent, often-white, often-male, often-young startups, to caricature). But I do believe that this kind of test would suffer from strong false negatives as many kinds of people who would otherwise be excellent programmers (or whatever other $job) are rejected for not doing well on $test.
That's why I worry about anyone who puts real stock in these kinds of tests - it is (generally) indicative of a kind of science blind spot in the user. People are critical and skeptical of all kinds of other scientific claims, but psychological measures tend to get a pass.
I imagine the fact that a lot of us tend to score well on IQ tests has a lot to contribute here. When I found out how many standard deviations above the mean my IQ score represented, I was pretty excited about how awesome I was. When I realized that the test was probably baloney, I had to figure out more interesting ways to shore up my self-esteem. Personally: I bake bread and fish for compliments on the quality of my sourdough.
I learned, as an adult, that I had taken an IQ test while young and had scored in the 99.97th percentile. I think that I got that not everything benefits from what IQ tests measure. That said, I had already started seeing great success in a knowledge industry. It is clear that you also have great linguistic intelligence. I don't think you could completely dissociate your IQ test results from that intelligence.
The fact that certain groups of people tend to test well (or, taken conversely, that there are so many reasons why otherwise "intelligent" people don't test well) just exacerbates this problem and I think the central claim is still reasonable: things like IQ tests aren't necessarily strong predictors of tech skills, are dangerously close to pseudoscientific when misapplied, and have enough other theoretical problems that they should probably not be utilized during a hiring process.
As to my own linguistic skills, I contend that a lifetime of reading as well as a MS (in computer science), a Ph.D (in cognitive science and education), and ten years afterwards in academics including a professorship has prepared me as a writer. Did I have some initial Potential that gave me a head start? Maybe. If nothing else, I had the head start of my general introvertedness and a love of both reading and geekiness. A short test that purports to measure a fixed potential somehow inherent in someone is going to be pretty flawed.
But my "I don't doubt" phrase was more along the lines of ceding a central point: there probably do exist certain inherent characteristics that vary between people and provide some kind of predilection or head start. I'm just not yet convinced that the scientific community has really identified them yet, or that they're really able to effectively measure them yet. Instead, we get proxies that have a very high false negative rate, especially among otherwise-marginalized groups.
Anyway, four paragraphs is probably enough here.
I had an MBTI test pretty much forced on me a while ago (senior engineer would be unhappy if I didn't take it). I argued very forcefully against it, saying it was like astrology, that it made me feel like the company was asking me to participate in some religious ritual when I was an atheist and so on. I'm not in that company anymore and one major reason I ran was that they allowed pop psychology like that to dictate decisions.
The worst I've seen was where I had to do 100 questions and half were spatial intelligence (gameable and wtf). It looked like an incredibly expensive piece of software. Didn't get the job, thankfully, but the company's now known for having a policy of not being able to talk about work out loud if you're in IT. The other places that gave me tests had dissimilar, but equally insane workplace policies.
Rambling list: -At some level, the person interviewing you (or management) doesn't trust their intuition to find the right candidate.
-It's an easy way to throw someone out of a pool despite all else. There's a reason it's illegal to do IQ tests.
-Mental health issues such as dyslexia or ADD.
-I've taken maybe five intelligence tests for work and none of them have been the same. Where's the standard?
-Companies have a weird tendency to use puzzles such as the "three light bulbs" one. All that does is test if you've heard it before. "I want to see how you think" makes no sense when the question's on paper and there's 100 questions just like it.
-A lot of these tests are gameable since you can just practice beforehand.
-Test anxiety sucks.
The company who gave me the test offered me the job after a very brief interview, in which they said the test was not important. It was very clear from their behaviour that they thought it really, really was (important) and they were about to hire some sort of genius. I ran. I'm still running.
In my current position, at Big Financial Corp, I had to take an "aptitude test". There were three parts of it: literacy, numeracy, logic in that order. I suck at arithmetic so after the literacy bit I wanted to get straight to the cool and fun logic puzzles at the end, but the time we were given was like a third of the time the test was meant to take (allegedly, they wanted to see how well we perform under pressure) and I wouldn't have the time to do all three parts equally well. So I thought, screw it, I'll approximate. I clicked through the numeracy test mostly at random (I concentrated clicks on the first three options), hoping to get a 50/50 score and dove right into the logic test at the end. I had great fun and I got hired after all so it can't have been that bad. They haven't told me how well I did because they don't like to um, foster antagonism. Or something.
Point being: IQ tests are stupid. You don't need to have a high enough IQ to pass them. You just need to be smart enough to beat the test. So, useless.
http://iq-tests-for-the-high-range.com/statistics/iq170.html
Also, please read about IQ and rational thinking: http://www.scientificamerican.com/article/rational-and-irrat...
>If they give you an IQ test or similar, run.
I ran across this study a few months ago that says IQ-like tests, structured interviews (canned questions), and work-sample evaluations were some of the few useful interview tools for selecting good employees [1].
[1]. http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...