The pymetrics test mentioned in the article involves among other things, clicking on a balloon till it pops and pressing the keyboard as many times as possible in a minute. As much as people love to hate on leetcode-style interview programming questions, they are at least remotely related to performance on the job.

Personally, I've found it to be a good filter in terms of deciding where to interview at. Companies that have adopted these pop-psychology tests (management consulting companies and big banks like BCG, JP Morgan) signal that they've let non-technical upper management hold sway over recruiting in comparison to tech-centric companies/hedge funds etc. that seem to have more rigorous recruiting standards.

> Personally, I've found it to be a good filter in terms of deciding where to interview at

Emphasizing this point for others. This is the best way to approach it IMO.

The company should work to impress you as much as you work to impress them.

Got a call from a recruiter that was looking for an oddly specific combination of skills. Me, so it’s Company X? He was flabbergasted. I gave 3 months notice. I was literally the only person in the world with that combination of skills. So I asked how much, went to bosses, I’ll stay on longer if you pay me that rate.

Good money for a few months.

I suppose that speaks positively of the recruiter's ability to find the right person for the job!

This. I got headhunted for a very very specific skill in my city. Asked them if it was company X? “How did you know??” “I wrote that rec”. I then proceeded to work directly with the recruiter to find a candidate and got an easy $1000 for about 4 hours of total work.

I'm curious about what the combination of skills was ?

Obscure mainframe tech and ruby, VB. About 15 years ago.

> The company should work to impress you as much as you work to impress them.

This becomes more true the further in your career you go. But for entry level positions it's really just a numbers game.

that's the balloon task, it measures willingness to take risks in contexts of ambiguity (the chance of it popping is not known). the button press task is probably used to normalize response time data, since some of the between person differences in average response time is due to (probably uninteresting) differences in motor-control of the neuromuscular system. were there any other tasks like ones that asked you to identify the color of the text regardless of what the word said (e.g. a blue word "red")?

The "willingness to take risks" is probably also "willingness to take crap" by an unemployed or underemployed person. A gainfully employed person may move on when the pseudoscientific video games come up during the screening interview. It is a risk to go work for such companies.

I think I would dispute the characterization of this as pseudo-scientific. The analogue balloon risk task is well validated, if in some ways poorly understood. That is, while standard economic risk models don't map onto it very well, it does correlate fairly substantially with real-world risky behaviors (so again, not economic risk where there is variance of outcomes, but more like the kind of risk taking where there are predictably bad outcomes, like trying heroin).

On the other hand, the use of these measures by big corps very well might be pseudo-scientific. I generally object to psychometrics being applied to prospective employees; it feels very dystopian to me. I also know that these measures require more subtle interpretation--and caution--than you're likely to get from anyone in HR, especially when we are talking about making inferences about individuals rather than groups of individuals.

Why you are being down-voted is beyond me. Your response is reasoned, clear, and open-minded, as well as informed. A citation might improve it, but as it stands, its a fine exemplar of the kinds of comments we should be promoting here on HN.

I’m an IO psych at a large tech company that evaluated pymetrics against job performance and there was no relationship, just near 0 correlations so recommended not using them and we don’t. These AI tools are not transparent enough in explaining their outcomes and lack what we call face validity or job relevance. A coding test is at least a good filter at the top for entry level roles with large pools because it has relevance and false negatives aren’t as much of a concern (but we test still for disparate impact) otherwise for most roles a structured interview is the best option.

This is why tech companies are good. Our output basically said "this is entirely indistinguishable from random choice," and we got the response "Great! Can't get sued over randomness!"

Even fairly strong correlations break down on restricted ranges of inputs. Eg, if you're hiring between 80-90th percentile candidates (top ones get better offers, lower ones get filtered out), yes, the correlation will get swamped by noise; not so much if you are hiring between 0 and 100.

Based on how these things go, it's almost a certainly that this will be found to be discriminating in some way that is unethical/illegal within the next few years.

> (1) It was not clear to us, with any level of certainty, whether either tool did anything more useful than filter out people who don't want to take the test / don't trust "AI".

There's at least one way this is beneficial for the employer: Desperate, obsequious job-seekers will be more willing to tolerate the degradation of being judged by a computer. It's perfect for finding employees who will ask "How high?" when you tell them to jump.

I applied to a few jobs at United Healthcare and they kept sending me HireVue interview invitations. Every time I rejected them and when prompted for feedback I said I don't want to be evaluated by a computer. Never heard anything from them (and didn't expect to).

How do you find good psych people? Showing my bias but it’s a genuine question. I remember in school all the psych and social psych folks being total jackoffs in blowoff classes.

How do you find people you can take seriously? Who don’t just quote psychology today and act like it’s insight? Like I know the field is professional and has skilled professionals - but how do you find them?

Well, first you put out a job ad. Then because the psych people are so numerous, you'll be inundated with resumes. Since you don't know what to really look for, you decide that an AI from some company will help out with that. So you use an AI to hire people to get rid of the AI, except in the case where you hire people to get rid of the AI, because you don't know how to hire those people. ;)

So how do you find the right psych person to monitor the psych hiring AI?

I actually built an AI to do that for you

Who's gonna monitor that AI?

It made a strangely compelling argument that it would be fine if it forked off a subprocess to monitor itself.

But it also has to fork off a subprocess to hire that subprocess from a pool of randomly generated subprocesses

For hiring and assessment design look for people with at least Master degrees in Industrial Organizational psychology. PhD isn’t necessary because it’s an applied field and there aren’t many applied PhDs so work experience is more important. Most start out in public sector which is much more rigorous in their interviewing and defining roles than the private sector is. SDSU and SFSU have great programs in CA and their professors can connect you to their alumni networks. SIOP is the professional organization. IOPredict, Biddle, and RocketHire are some consulting firms.

You don't hire right out of university, but middle-to-senior people who have established themselves in academia.

That makes sense - thank you.

> I had people telling me they used supervised learning because they had a team of supervisors in India

Love it.