So what you get by discouraging a broad diversity of personalities is a mix of people who actually have those specific characteristics and sociopaths. The latter group ends up running the company, and of course they issue utterly sociopathic "psychometric evaluations", which are basically just encoded discrimination against entire classes of people, regardless of job fitness, because the people up top are such incompetent leaders that they need subservient workers to obey their crazy orders without question.
- helpless
- depressed
- jerks
The top one has about %50 shot of being a character flaw, eg, no amount of confidence-building will help.
The middle one has the most hope, if the talent potential is significant. The best in the field can be painfully shy as well as depressed. The problem is that depression (not shyness) hurts morale. [1]
The last has a %75 chance of being a lost cause. Impossible to manage as they are incredibly disruptive to morale and productivity. [2] Their survival is often linked to mastery of politics, which inculcates their position.
[1] http://m.fastcoexist.com/1679208/a-sad-worker-is-a-bad-worke...
[2] The No *sshole Rule by Robert Sutton
When you have worked with someone that screams at their coworkers, can barely feed/clothe themselves or refuses to lift a finger to help you... then you will know how much time and emotional effort these behavioral traits can waste.
I cringe whenever I read job ads that are aiming for a culture fit: "we only want to hire harmonious, independent, extroverted employees;" when in my experience, the best employees do not hold strongly to those ideals, but neither to do they hangout at the other extremes.
There is the another option. Make something people want and build it up slowly, on the side. If you do, you may free yourself from being at the beckon call of other masters (other than users). Others have, so it's possible. We are each masters of our own fate.
Peace.
Only someone without life experience would not read between the lines that buckets are ideals and need the explanation that hiring is holistic.
To then make a straw man fallacy out of the opposite of an ideal is to be inconsistent: you can't have it both ways.
- people that put their percentage symbols in the wrong place - People who capitalise their sentences.
- People who spell 'capitalize' as 'capitalise'
That's the value of these tests (when properly applied): They allow an evaluator to make statistically accurate predictions of specific types of behavior. The key is the two words: statistically and specific. A single composite score will not tell you anything, but the idea that there's a 68% chance that the person you're about to hire is a schizophrenic kleptomaniac should give you pause.
Sorry, I don't believe "psychometric tests" can do that either. At least not at the level that we're talking about: distinguishing an otherwise qualified person from someone who's got some sort of psychopathology.
I don't believe in employment discrimination on the basis of mental illness. After all, John Nash was schizophrenic.
So what your letting in is a subgroup of people who pass, and psychopaths.
In these tests, they know what to say, what to do, to give a certain impression. These tests can't catch them, because they are not honest, and lack integrity(No consistent values, only ones that benefit them).
There are certainly psychopaths who are hopeless at concealing the fact. These are typically the less smart ones, so you wouldn't meet them; they are more likely to be in and out of prison than your office.
There are also psychopaths who are just about perfect at pretending not to be, and have no intention of ever doing otherwise. You can work with one of those for decades, and never notice a thing unless something extreme enough to make 'acting normal' seem a long-term liability happens.
I don't remember where I got the number, but apparently something like two-thirds of the regular army still end up unwilling/unable to aim properly when a combat situation actually happens. That's from the Great War, though; procedures may have gotten better since.
You'd think that, but a quality psychometric test can't be gamed if it was developed in-house and is only taken once by the candidate. This particular company might not have had one of those, however.
Professional personality tests can't be gamed - you also can't find these on the internet, because they're very strictly copyrighted.
Instead of trying to get rid of questions that are gameable, the psychometrists design the test with statistical consistency - things as obscure as, say, what color the candidate answers in a multiple choice question out of orange, yellow, blue and red can be used to correlate with other questions (not literally, but to a candidate it would be equally obscure and seemingly innocuous as a question). If a candidate answers one question in what they perceive to be the societal ideal, this will be exposed in other questions where it's not possible to cross-consistently game the question unless you've read and thoroughly understand a manual of psychology or psychometry.
Tests designed this way allow for a certain amount of "gaming" by candidates before it crosses a statistical threshold, at which point it essentially tells the proctor, "The answers are so inconsistent that the candidate wasn't being honest." and you have to throw out the whole test (which in the context of hiring, means rejecting the candidate).
The reason why this works is because while people will answer "Would you consider yourself hard working?" with "Always" or some other unrealistically gamed superlative, they won't realize that other questions that are seemingly unrelated are highly correlated with that quality - if you answer yes to one and no to another, you probably lied on the transparent one, with high confidence.
Tests like this[1] have been around for so long that they consistently get better, though there are some valid criticisms of them being easier for non-minorities (for a variety of reasons). They are used in clinical and professional contexts, and while the quality personality tests will theoretically be consistent across multiple-test takings (in other words, results are relatively immutable), in practice you should probably avoid giving one individual a lot of exposure to the same exact test twice.
Hope that helps.
[1]: http://en.wikipedia.org/wiki/Minnesota_Multiphasic_Personali...
The MMPI is harder than the Myers Briggs, but not that difficult. Additionally its really only optimal for clinical samples, at which it is very good.
However, everyone in the field knows that these tests can be gamed, the only open question is how many people game them consistently.
You can probably estimate the proportion of social desirability exhibited in job interviews by comparing to non-job situations (such as a sample matched on all relevant covariates (whatever they are) from the general population.
Nonetheless, believing in personality tests as an accurate indicator of personality is as misguided as believing that Facebook represents the social graph of all its daily active users accurately, i.e., somewhat misguided.
And I am aware of lie scale, and they are trivial if you actually read the question. Protip: If a question says always or never, its probably designed to trip you up.
I do agree that personality tests are more accurate than this thread makes them out to be, but they are certainly not as useful as your comment implies.
Is there anything you've got offhand that I can read further about this? I didn't know you could actually game them.
Despite this, I have ran many surveys back when I worked in academia. You can detect some of this stuff with Guttman errors, but these are not often used, and as long as you are consistent, its very difficult to spot.
Can you provide a deeper example? I'm honestly curious - what sort of question is innocuous enough to be answered honestly, yet useful enough to provide information? (i.e., do slackers like the color yellow or something?)
http://www.physicschick.com/pole/2012xxxx/psych.txt
By googling those, you can go deeper into the rabbit hole.
A acquaintance of mine is finishing his dissertation on faking on personality and integrity tests, so I'll know more on that soon.
However I question your absolute statement. What man can create, man can circumvent. (How's that for an absolute statement?)
Would you say that the people who created the test cannot game it?
You should also read upthread, what the actual psychologist said. It clarified my comment really well.
Oh come on. This was posted in a different comment:
http://www.physicschick.com/pole/2012xxxx/psych.txt
The following are very obviously the "correct" true or false answers to these questions from the MMPI-2:
T * My mother is a good woman.
F * Evil spirits possess me at times.
F * There seems to be a lump in my throat much of the time.
T * At times I feel like swearing.
T * My hands and feet are usually warm enough.
F * Ghosts or spirits can influence people for good or bad.
F * Someone has been trying to poison me.
F * Everything tastes the same.
F * Someone has been trying to rob me.
F * Bad words, often terrible words, come into my mind
and I cannot get rid of them.
F * Often I feel as if there is a tight band around my head.
F * Peculiar odors come to me at times.
F * My soul sometimes leaves my body.
F * When a man is with a woman he is usually thinking
about things related to her sex.
F * I often feel as if things are not real.
F * Someone has it in for me.
F * My neck spots with red often.
T * Once in awhile I laugh at a dirty joke.
F * I hear strange things when I am alone.
F * In walking I am very careful to step over sidewalk cracks.
F * At one or more times in my life, I felt that someone
was making me do things by hypnotizing me.
Again, I'll reiterate: can they be gamed? Yes. Is it likely? Emphatically, no.
Are you referring to the MMPI or something similar? Or to Unicru and Kenexa? Unicru seems to ask much more prosaic questions than MMPI.
And yes, when I speak of gaming, I am assuming prior knowledge and preparation - but not necessarily inside knowledge.
Do you have any support for this where the employees are both technical and creative? I'm obviously thinking about software development, but it should apply to any kind of engineering or technical job where the employees need to have a deep technical knowledge, know how to apply that knowledge practically, and have to use that knowledge to create new solutions to new problems.
If so, under which circumstances are they used? For a graphic designer, the natural "test" would be for fellow graphic designers and potential managers to look at an applicant's work samples, or to ask them to produce one. This method directly tests the applicant's ability to do the type of job, although there is no objective metric. You are relying on people's subjective assessment. How do cognitive ability tests compare?
Here is a good article you can read on the subject: http://www.unc.edu/~nielsen/soci708/cdocs/Schmidt_Hunter_200...
Related to this, I think it's important to consider that this correlation between g and job performance is conditioned on the fact that the person applied for the job. That sounds trivially true at first, but it means that the applicant felt like they were competent for the job (in the best case; in the worst case, it meant they felt that they had a chance of appearing competent at the job). In other words, what we're saying is, "Of the people who thought they could do the job, the smartest ones tended to do the best."
But if our candidate pool was everyone, I'm skeptical that g would still hold as a good predictor. I think I'm a bright guy, but I'm pretty sure I'd make a terrible nuclear engineer. And with that in mind, we may need to keep the non-g related selection around to prevent such a situation.
You are right to think that the results would be different if the test was just given on the general population. It's an academic consideration that tends to resolve itself in the field.
And you'll never hear me, or anyone else, suggest that g should be the only predictor for just the reason you describe. Biographical data (e.g., years experience, work history) is much better first hurdle.
I've also read that two of the strongest predictors of performance in school are your parents' performance in school, and (after controlling for that) your parents' income.
Do studies of general intelligence usually control for the impact of parents' education and income?
I'm intrigued, is the "bias" because the test is unfair to one racial or more racial sub-groups or because the test is "fair" and that is how they actually perform or is it a language thing.
I do reasonably well on standardised IQ tests but I suspect if I did a German one I might struggle.
This is a quick overview of the topic: http://www.siop.org/_Principles/pages31to34.pdf
An interesting thing is that the predictive bias is reduced for open-form response questions vs. select-one tests. It's an indication that there is more at work than just subgroup differences.
One of the people looking at the results then looked at the breakdown of questions by group and noticed immediately that questions like "The cup goes on the a) saucer b) floor c) table d) shelf" where consistently "wrong" (correct answer was a) for the poorer group at which point he realised that working class children drink tea from mugs and saucers where a middle class thing.
The story might be apocryphal but its stuck with me since I was 12-13 whenever I run into any kind of standardised testing/results.
How do we know the tests are fair? For a given test score, life outcomes like income and criminal conviction rate are the same.
As, I'm sure you can tell, this isn't specifically for technical and creative employees, but in general it appears that structured interviews, IQ tests and work samples significantly out perform unstructured interviews.
Why?
Having read Thinking Fast and Slow and been convinced by this book's many useful views, I would agree that a simple questionnaire-based ranking is actually better than any subjective assessment of candidates. And it should be as neutral as possible, ie not letting one's "first impression" influence the ranking (because it is almost always based on irrelevant physical features).
But even then, there is no reason for this neutral evaluation to become worthless if not 100% consistently applied. There is the broken leg case: If you want to evaluate the probability of someone going to see a movie tonight, you just base yourself on simple statistical facts (how frequently people go to the movie in this country), and should not try to infer more from subjective context, except if the guy in question broke his leg this morning.
These tests and evaluations help much in reducing system bias due to halo fallacy, framing effect, and even time of the day for the evaluator (it has been proven juges are more lenient after lunch!), but they still are only helpers, they do not need to be decision-blocking.
Statistically, you've created a model for consistent application, and if you don't consistently apply then the anticipated value of the tools you're using is now in question as the model has a large source or error involved now.
I don't mean to say subject evaluations have no place in the process. Think of each part of the selection process as an independent stage. The candidates can pass the objective tests, perhaps with a minimum cutoff score, then pass the subjective judgment test. Or vice versa (though that may limit utility further).
I can't speak to your second point, but as to your first point, I should note that law firms themselves definitely do not use such HR-driven hiring processes.
Also worth noting: sometimes less rigorous way is the better way. Developing validated tests doesn't work as well with smaller numbers, and can be expensive. There are also considerations to be made for the selection ratio (candidates selected over candidates applied). The utility isn't always worth it, and I shouldn't make it out to seem so.
Let's assume for a moment that when a person is told to do something, and they disagree, 90% of all employees are typically wrong when they disagree and 10% of all employees are typically right when they disagree.
We are considering now just engineering type jobs (since that's what the article is about).
Now consider anti-authoritarianism: 90% of the people who are anti-authoritarianism will perform terribly worse than the typical population (since they will tend to go with their mistaken belief of what is right, or be obstructionist when poeple want to do it differently, etc). They will be your least valuable employees.
The remaining 10% of those scoring high on anti-authoritarianism are among the most valuable of your employees, as they won't allow their teammates to go down a dead-end.
So anti-authoritarianism is highly correlated with poor performance (even trouble-makers) but rejecting all applicants based on this will keep you from getting some of the most valuable employees.
Here I have a problem - which job performance? Different jobs are obviously demanding different qualities - highly creative but impatient person may be an asset in job requiring instant creativity but a liability in a job requiring steady repetitive tasks and constant attention. So the test has to be matched to a position. But in the original article not only HR gave the same test to everybody, people who decide on positions don't even seem to have information or input on any correlation between position and test results. Given this, I highly doubt such application of testing can have any meaningful correlation with job performance.
Job performance can be defined many different ways, and the most common in field studies is what is most common in the workplace: supervisory, and possibly peer, evaluations. It's not the perfect criteria, but the "perfect criteria" for many jobs exists only in the theoretical. When appropriate, "work performance" is also defined as sales numbers, widgets produced, time-to-complete, etc. It just depends on the study.
> So the test has to be matched to a position.
In many cases you're right (e.g., work samples), but not always. g-centric tests have demonstrated high validity across jobs and contexts. Tests for cultural fit (in which the desired outcome is typically workplace satisfaction and tenure) are typically validated across a wide range of jobs, and so they are applied across those jobs. Things like integrity tests and personality tests can have meaningful validity across jobs, even without seemingly job-relevant content.
Properly validated psychometric tests are valid predictors of job performance (by which I mean objectively and factually correlated with actual work performance), and a properly constructed work-relevant test will be on the whole will be more valid than subjective judgments from most technical professionals. Of course there will be hits and misses, but the consistent application is the key to its use. If you don't consistently apply it, it becomes worthless.
But, let's throw that out of the window for a second because in this particular anecdote I don't think it's relevant. This test they were using seems more like a cultural fit, specifically a person-organization fit test. These tests are less about predicting job performance than they are about predicting organizational commitment and ultimately tenure [1]. Surely we can agree that those things are important too.
But, let's again say that this candidate's aptitude is so impressive that we don't mind an increased statistical chance that his or her tenure may short. The real issue is that legally you are obligated to apply the same set of criteria all of your employees for a given role (not necessarily the exact same position, but comparable positions). If the company is applying cognitive testing, personality testing, drug testing, person-org. fit testing, etc. to one group of applicants, you now cannot fail to adhere to that criteria because it is an invitation for legal liability, however seemingly-spurious.
Having said that, it's important that your selection processes have been validated previously, or at least are in the process of being validated (which provides some protection). I also have a big concern with the linked situation because: 1) it's a cultural-related test, which means the possibility to show cultural-related bias is high and 2) they were unwilling to make language accommodations, which seems unnecessarily rigid when native-level English skills are likely not a core component of the job. Point #1 can be accounted for in some shape or form (at a minimum through a low cutoff score), but Point #2 is more problematic.
Anyway, there is a reason companies adopt "Big Corp" characteristics as the scale. The primary reason is for legal compliance through standardization of process, and the other is that data is supportive of the validity of objective predictors of job success relative to subjective judgments.
I realize I may be opening myself up to some criticism and scorn from HN crowd for seemingly representing "Big Corporate" or acting like a Bob from Office Space, but so be it. Despite what it may seem, there are often smart people using evidenced-based processes driving seemingly asinine HR processes that drive you crazy. And sometimes there aren't, and it's just a big pain in the ass for poor reasons.
--
[1] PDF of meta-analytic findings about person-job/org fit: http://nreilly.asp.radford.edu/kristof-brown%20et%20al.pdf