News for hackers. Stuff that matters.
Re:Software Raises Bar for Hiring by tokenadult on Saturday June 02, 2012 @02:06PM (#4058682)
The underlying problem is that most companies rely on hiring procedures that are empirically demonstrated to be suboptimal. Companies rely on industry tradition or the gut feelings of the boss rather than on research to decide hiring practices. The review article "The Validity and Utility of Selection Models in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings"

http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...

sums up, current to 1998, much of the HUGE peer-reviewed professional literature on the industrial and organizational psychology devoted to business hiring practices. There are many kinds of hiring screens, such as resume reviews for job experience, telephone interviews, in-person interviews, checks for academic credentials, and so on. Most of the most commonly used means for screening job applicants, such as those mentioned in the submitted (like matching job titles in previous positions) are basically useless. There is much published study research on how job applicants perform after they are hired in a wide variety of occupations.

The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well (but only about at the 0.5 level, standing alone). One is a general cognitive ability test (an IQ-like test, such as the Wonderlic personnel screening test). Another is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. Each of these kinds of tests has about the same validity in screening applicants for jobs, with the general cognitive ability test better predicting success for applicants who will be trained into a new job. Neither is perfect (both miss some good performers on the job, and select some bad performers on the job), but both are better than anything else that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.

For legal reasons in the United States (the same consideration does not apply in other countries), it is difficult to give job applicants a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of a hiring process. The Griggs v. Duke Power, 401 U.S. 424 (1971) case in the United States Supreme Court

http://scholar.google.com/scholar_case?case=8655598674229196...

held that cognitive ability tests used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring process had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Note that a brain-teaser-type test used in a hiring process might be illegal if it can be shown to have disparate impact on some job applicants and is not supported by a validation study demonstrating that the test is related to successful performance on the job. Companies outside the United States are regulated by different laws. So don't set up stupid HR procedures like those described in the submitted article, even if those stupid procedures can be automated. Instead, figure out how to give applicants a work sample test, or figure out how to validate a general cognitive ability test for the applicants you desire to hire. Automating USEFUL processes can be helpful. There is no reason to automate a process that is not worth doing.

Re:Software Raises Bar for Hiring by wisty on Saturday June 02, 2012 @10:07PM (#4059813)
> Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company).

It's weird, as the actual policy was "requiring a high school education or passing of a standardized general intelligence test".

Surely there's a lot of African Americans with a high IQ, but no degree. Wouldn't this case mean that requiring a certain level of education is worse than requiring a high IQ?

Re:Software Raises Bar for Hiring by lifeisstillgood on Saturday June 02, 2012 @03:52PM (#4058992)
In the UK the govt wants to implement a report commissioned from a hedge fund friend of the PMs - one of the key "political" recommendations was to introduce no fault firing - making firing staff ridiculously easy because that is obviously want is wrong with UK PLC. The elephant in the room is that if you have hired large numbers of crap staff and you can't get rid of them because it will cost too much or they will all sue, then clearly your past hiring practises were wrong.

It's interesting how we still run to the government to solve our own past mistakes.

Re:Software Raises Bar for Hiring by learc83 on Saturday June 02, 2012 @02:32PM (#4058756)
>like the SAT

I would love it if companies hired people like colleges select students.

I'm very good at interviews, but I still hate doing them. If would love it if I could write an essay and send them my SAT scores without going through that awful process.

It would probably also weed out people who are just inordinately good at interviews.

I don't overtly try to exaggerate, but a few times I've been offered a job where I thought in the back of the mind that I might not really be qualified. To wit, I thought my interview skills, not my coding skills got me the offer.

Re:Software Raises Bar for Hiring by MaxGabriel on Saturday June 02, 2012 @02:44PM (#4058785)
I get what you mean, but colleges are probably the worst analogue for successful hiring practices given selection based on legacy status, wealth, minority status, athletic ability, etc
Re:Software Raises Bar for Hiring by mikeryan on Saturday June 02, 2012 @06:11PM (#4059343)
A note about the above. The article actually states that the best indicators of performance wasn't actually just a GMA (General Mental Aptitude) test or just a trial but in fact the combination of a GMA test and a integrity test or a GMA test and a structured interview. You're not off the hook by just testing well.

Of the combinations of predictors examined, two stand out as being both practical to use for most hiring and as having high composite validity: the combination of a GMA test and an integrity test (composite validity of .65); and the combination of a G M A test and a structured interview .(com posite validity of .63).

Re:Software Raises Bar for Hiring by entangld on Saturday June 02, 2012 @08:57PM (#4059702)
Resumes can be VERY misleading.

We just finished interviews last week and a few people who looked perfect on paper were not even considered after the interviews.

I was one of two interviewers and I only threw softballs.

Re:Software Raises Bar for Hiring by derekerdmann on Saturday June 02, 2012 @02:59PM (#4058821)
But it wouldn't weed out the people who can do well on tests but can't apply their skills to real problems. Part of the interview process is getting a better look at the person behind the resume, and mailing in test scores defeats that.

Also, all the SAT does is see how well you can take a 4-hour test.

Re:Software Raises Bar for Hiring by mahyarm on Saturday June 02, 2012 @03:02PM (#4058831)
Some people are also really good at essays. It's also like talking to a wall, and it would filter out many effective immigrant workers.
Re:Software Raises Bar for Hiring by orijing on Saturday June 02, 2012 @04:28PM (#4059084)
Wait, I'm confused... Are you saying that it can be illegal to have hiring practices that are suboptimal? For example, you mention that brain-teaser tests might have disparate impact on some applicants--and they're also not that effective anyway.
Re:Software Raises Bar for Hiring by WildUtah on Saturday June 02, 2012 @05:52PM (#4059294)
Diaparate impact means disparate racial impact (or disparate against another protected class, but usually racial).

So if you hire a smaller fraction of any racial category of applicants than there were in the pool of applicants or in the workforce of your region, then your hiring method has disparate impact. If you use an objective test that contributes to that disparate impact, you have to be able to prove by thorough studies that it was the best method to pick staff for the specific job you are hiring for. Such studies start around seven figures and won't work for positions with multiple responsibilities or difficult to measure outputs.

Re:Software Raises Bar for Hiring by jaredsohn on Saturday June 02, 2012 @07:46PM (#4059565)
>So if you hire a smaller fraction of any racial category of applicants than there were in the pool of applicants or in the workforce of your region, then your hiring method has disparate impact.

Is it necessarily because of the hiring method, though? Couldn't it be possible that there is something about the company as a whole that causes people in a protected class to avoid applying to the company? (For example, if the company has racist ads.)

Re:Software Raises Bar for Hiring by thaumasiotes on Saturday June 02, 2012 @05:26PM (#4059224)
I believe you're parsing it wrong. Read "Note that a brain-teaser-type test used in a hiring process might be illegal if it can be shown to have disparate impact on some job applicants and is not supported by a validation study demonstrating that the test is related to successful performance on the job" as

"Note that a brain-teaser-type test used in a hiring process (might be illegal if [both] it can be shown to have disparate impact on some job applicants and [if it] is not supported by a validation study demonstrating that the test is related to successful performance on the job)",

not "Note that a brain-teaser-type test used in a hiring process (might be illegal if it can be shown to have disparate impact on some job applicants) and ([furthermore] is not supported by a validation study demonstrating that the test is related to successful performance on the job.)"

As I read the sentence, a general brain-teaser is supported (by all the research showing that tests of general mental ability are predictive), but it is not legal unless you've done a study specifically for that brain-teaser.

Re:Software Raises Bar for Hiring by TheCount on Saturday June 02, 2012 @05:58PM (#4059311)
To explain the issue further, tersely, at the risk of being downvoted, in America, the average score among blacks is a standard deviation lower than that of whites, on IQ tests, that is, 15 points. Given that IQ is normally distributed for the most part, using IQ as a factor in hiring will reduce the proportion of blacks who pass the 'bar.'

This gap may diminish if you're looking at a pool of people who passed some previous threshold, eg they have a degree in math. But even then, some of the gap remains, due to the shape of normal distributions. Think of the heights of men and women, and how even if you looked only at people six feet and taller, the proportion of women in that pool who just barely meet the threshold of 6' would be much higher than the similar group of men; despite the cutoff, the average height of the women would still be less than the average height of the men, within that group. Draw overlapping bell curves to see this visually, and analyze the areas under the graph of each distribution, to the right of a given cutoff point.

We do have a law against invidious racial discrimination. The current implementation of the law states that in practice, if some selection mechanism has a passing rate for one minority that is less than 80% of that whites, that is evidence of discrimination in the absence of validation of that mechanism. This is known as the four fifths rule. A test in which 10% of whites and 7% of blacks passed would fall afoul of this law. Say you set a 115 IQ threshold. That is +1 S.D. from the white mean roughly, and +2 S.D. from the black mean, meaning 16% of whites and 2.3% of blacks would pass, if drawing from the general population. That wouldn't even pass a 'One fifth rule,' let alone a four fifths one.

And validation is no sure thing either - police and fire departments have had these tests validated and still have the results thrown out regularly, due to disparate racial impact, i.e. relatively few blacks and Hispanics (average IQ: ~90) passing the test.

On a related note, explicit racial and gender quotas are more efficient than the race- sex- and IQ-blind, approach in use today. A quota would mandate in effect different thresholds for different groups, but it would allow one at least to use this selection mechanism. Within given groups, one would obtain more efficient outcomes. The problem with quotas is that they make racial preferences explicit, while abandoning such selection methods entirely is only an implicit, de facto racial preference, a rather subtle and hidden one at that.

Re:Software Raises Bar for Hiring by tokenadult on Saturday June 02, 2012 @11:16PM (#4059929)
There is updated research on the issue of differing distributions of scores between the groups of people characterized as "black" and "white" in the United States. The score gap is narrowing, according to an important review article published this year by the American Psychological Association,

http://www.apa.org/pubs/journals/releases/amp-67-2-130.pdf

and investigating how the gap arose in the first place has been very illuminating about the effect of early childhood environment and formal schooling on the development of IQ. But, yes, the reason the Supreme Court ruled as it did in the 1971 Duke Power case is that there has been a difference in the distribution of IQ scores among "white" and "black" people in the United States, and also an odious legacy of efforts to deny employment opportunity to black people, so if a company has a particular hiring process, it had better make sure that the hiring process will "bear a demonstrable relationship to successful performance of the jobs for which it was used."

See the writings of James R. Flynn, particularly

http://www.apa.org/pubs/journals/releases/rev1082346.pdf

http://www.iapsych.com/iqmr/fe/LinkedDocuments/dickens2006a....

http://www.iapsych.com/iqmr/fe/LinkedDocuments/dickens2006b....

http://www.iapsych.com/iqmr/fe/LinkedDocuments/flynn2007c.pd...

and

http://www.iapsych.com/iqmr/fe/LinkedDocuments/flynn2010a.pd...

for background on the narrowing gap between "race" groups in the United States.

Re:Software Raises Bar for Hiring by cousin_it on Saturday June 02, 2012 @06:48PM (#4059427)
Great explanation! Just one small nitpick: IQ is not "normally distributed for the most part", it's fitted to the normal curve by definition.
Re:Software Raises Bar for Hiring by entangld on Saturday June 02, 2012 @08:51PM (#4059692)
How is this a black vs. white issue? There are plenty of smart people of all races who are unemployed.

Like the article stated, it's due to lack of experience in the job they are applying for, coupled with the skills gap. Employees are asking for more experience than applicants have.

Intelligence is not a factor if they are only looking for experience.