* Youth. People who have very recently studied these things in school, and use the same languages as the interviewers, have an advantage.
* Free time. People who have families (for example) might have less free time to study "Cracking the Code Interview" and such.
* Absence of anxiety. This disadvantages women, minorities, and people with psychological conditions that should be covered by ADA. Also, people whose financial situation is precarious will be more anxious than those who don't need the job, independent of which is actually a better candidate.
* Conformity. People who can recognize the flaws in a measurement technique, and who have the strength of character to push back against its application - both good qualities for a candidate - will self select out.
There's a lot of overlap among these, of course. There are better ways to measure "general IQ" and "grit" (which are both questionable concepts anyway). I've passed every such interview I've ever taken, but I refuse to administer them (despite the fact that my refusal has carried a quite tangible cost) because whatever benefit they provide is outweighed by their many flaws.
I resemble some of those categories, and I don't know if I would feel comfortable making the leap to correlate them to a some inherent reduced level of resistance to anxiety. That seems like a generalization which I feel that, on an aggregate level, seems unsupportable by data.
I think that determination should be on a case-by-case basis, as is currently done at universities.
Is any of that even controversial enough to require citation? How many Psychology or Sociology 101 textbooks should I cite? It's easy enough for those who don't feel this kind of anxiety themselves to brush it off, but for those who are less fortunate all those magnifiers can create an anxiety level that's quite debilitating.
(Only this past week I had experiences that challenged my assumptions about certain demographics (seniors) and how I would expect they would behave, and how they actually behaved. The lesson I learned was -- don't assume, always collect real data)
The white-male interviewer power dynamic has some basis in reality (I've experienced it occasionally, not all the time), but its effect on my interview performance may be less than 10%? (to throw out a number). I find I'm much more affected -- maybe 90% -- about (1) my competence in the subject matter and (2) how well practiced I am (for instance, I know the theory for a great many subjects but am unpracticed at some of them, so I tend to stumble and lack ease when it comes to demonstrating my subject matter knowledge in real time).
For different people, those percentages shift, and I believe in a way that is not obviously or necessarily correlated with their demographic (psychological conditions, yes, but also depends on which ones -- some don't affect anxiety). But all I have is anecdata so I'm not able to provide strong evidence one way or another so this is just my two cents worth -- and I do mean this in good faith.
My basic point here is that even 1% would be too much. If an interview process creates any inherent disadvantage for some groups, I'd say it deserves serious scrutiny. BTW, by "inherent" I mean beyond what can be addressed by bias training and such. That can enable interviewers to conduct any type of interview in as fair and kind a way as possible, but not to change the interview structure itself. If the structure is the problem, training isn't the solution. I think the in-person white-board algorithm interview is unavoidably weighted toward factors that have nothing to do with likely on-the-job performance, and thus should be avoided.
Going back to whiteboarding: I don't like whiteboarding myself, but to be fair it does measure certain dimensions quickness-on-feet, memorization abilities, ability to exude presence, fluency in language, etc. While these are laudable abilities on their own, I agree they might not correlate with overall on-the-job performance (but it depends on the job).
I guess "job performance" is this amorphous latent variable y that is correlated with a bunch of direct predictors x which we can measure, u which we can't measure, so we use proxy variables z to stand-in for them: y = f(x, u, z).
The worry is that some candidates, who may be bad for the job, but just happen to be good at these proxy dimensions (or train for them) might get the job; on the flip side, we may exclude certain candidates who are potentially good for the job but perform badly at the proxy dimensions. Whiteboarding measures the proxy dimension z.
Edit: oh look, an article on HN's front page on this very issue:
I'm sorry, excuse me? Are you saying that non-minority, non-women don't suffer anxiety? Your parent comment certainly seems to suggest that. Which, at a minimum, is flat out wrong. Educate yourself[0]. And then zoom out and ask yourself why it's not only permissible, but often lauded, to so flippantly say what you just said.
[0] - https://www.apa.org/about/policy/boys-men-practice-guideline...
To be fair, OP did say "That effect is magnified for anyone who is unlike their interviewers"
But, I do agree some evidence from OP would help their claims.
I wouldn't be surprised if minorities experience more anxiety, on average, in situations like an interview, though. I think it's established that imposter syndrome is more frequently encountered for example but it's too late here to go digging for evidence :)
No, and I doubt anyone would have read it that way in good faith.
And I'm not writing this as praise for your comment...
In other words, I’d rather work with a disadvantaged person who may appear rough around the edges but made it this far and has the “general IQ” and “grit” to do well on the programming problem, than the preppy white kid who has a lifetime of experience preparing for the task of exuding status and competency when answering behavioral questions or engineering case studies, but lacks the “general IQ” and “grit” to solve the programming problem.
I don't, any more. I'm at a company that enforces a very rigid structure that I don't agree with, so I simply opt out. I pay for it every review cycle. Ironically, the rigidity of that structure is explicitly intended to reduce personal bias, which is a laudable goal, and I believe it succeeds. Unfortunately, I think it just replaces personal bias with systemic bias.
When I did interview, which was a lot at times when I was in a leadership role at a couple of startups, my favorite interview technique was to let the candidate lead and I'd follow. If I wanted to ask about algorithms, I'd ask about one they'd used in a project they'd worked on. How did it work? What were its strengths and pitfalls? What others were considered? What bugs were found in its implementation, or caused by its use? Besides flipping the control dynamic of the interview, it often led to more interesting conversations. Highly recommend.
> what are some examples of better ways to measure general IQ and grit
IQ has been under a shadow since _The Bell Curve_ and I'm not keen on letting it back out. ;) If one must measure it, I'd say measure it directly with simple challenges (e.g. memory or pattern completion) or puzzles ... but even those are apparently fraught with cultural baggage and of questionable relevance to a knowledge-heavy domain like programming.
As for grit, it's often readily apparent from someone's resume. Were they self-taught, worked through college, or took a free ride? Did they stay with companies and projects through hard times and get promoted "in the field" or were they always the first rat to abandon ship? It usually only takes a few questions to figure out whether someone's a coaster or a fighter. Funnily enough, the people with the most actual evidence of grit are the ones least likely to have spent their time studying specifically for the interview. They were busy actually doing stuff.
It's a proxy for interviewers to jerk their ego.
The "amazing scientist" maybe was not a good culture fit, or, yes, the interview was botched.
Good candidates are so good that they often enough compensate for bad interviews. Sure it means we sometimes don't hire the best, but that's better IME than sometimes hiring a not so good candiate.
I do not see data and algorithm interviews as very specific technical problems.
I do not have a computer science background, and have been able to bring myself up to speed to the general level expected without too much hassle.
I don't think I'm particularly special. I just searched and spent some time learning this stuff over a few weekends. If you are serious about a career, I don't really think that this is too much of an ask.
It can be both
From my experience, when you have a big pool of candidates, the ones that pass not necessarily super stars, but they tend to perform at a relative stable level.
Interview for the skills you actually need. If the person isn't implementing algorithms and data structures from scratch, it's a shit question. Why would you ask questions that don't match the actual work they'll be doing?
If they will be doing this work, then obviously it's a fair question.
See, black and white.
> Why would you ask questions that don't match the actual work they'll be doing?
As an interviewer, and likely peer or manager of the potential new hire, I want to understand growth potentials as well, so I want to challenge you during the interview. In my particular situation, there are so many candidates, and so many mediocre ones, we needed to raise the bar, and it served us well.
Interviewers don't have choice here. Interviewers aren't free to come up with their own method.
So this is not a valid explanation.
Most companies require interviewers to pick a question from their internal 'question bank'
That's obviously not true....
> Most companies require interviewers to pick a question from their internal 'question bank'
Again, having worked at some fairly big and respected companies, this has never been the case.
I'm not interviewing for rote candidates. Everyone is different. Ergo, the questions are different. I could never imagine hiring senior developers and security engineers with questions from a "question bank".
If you're just doing boilerplate, you're probably getting very sub-par employees.
https://leetcode.com/discuss/interview-experience?currentPag...
Please take a look at these. I recently interviewed at FB and I got 2 questions in phone screen that were from leetcode with FB tag.
> Everyone is different. Ergo, the questions are different.
Facebook is running interview factory, they just don't have time to customize interview for each candidate. Their own recruiter told me to practise questions from leetcode tagged with facebook.
I agree with you re your reasons for not using a 'question bank' but thats just not the realty.
https://medium.com/@gameweld/the-case-for-the-private-techni....
Never attribute to malice that which is adequately explained by laziness and cargo-culting.
If I don't, I just stay put knowing I lucked out at a great company, knowing it would be hard to get lucky again.
[1] https://www.businessinsider.com/average-employee-tenure-rete...
1) one or two people in the discussion (interviewers) already know the perfect solution to the problem and are contributing as little as possible to the discussion.
2) the actual amount of allotted time to brainstorm a solution to the problem is realistically only ~10 minutes not the ~45 minutes you are allotted because of the time it takes to implement the solution with real code.
It would be nice if no one in the interview knew the answer to the problem before starting the discussion and this discussion would be pretty realistic aside from the lack of help from the interviewers. However, you cannot easily/fairly compare candidates with so many different questions.
Technical interviews would also be far more realistic if they allotted more time for problem solving but I imagine if the standard 45 minutes was bumped to over an hour, the only outcome is that more candidates would perform very well and that's not an outcome companies actually want unfortunately.
My personal opinion is that all of this should be scrapped in favor of seeing what the person has produced and having them partially implement a solution of something they have written in the past. It is a virtual impossibility that anyone could work on a project of any sort without being able to sit down for an 8 hour interview where they are responsible for partially re-creating something they have worked on in the past. To this day I still remember large parts of source code I've worked on 10 years in the past.
Why would this be the case--at places where the number of candidates far exceed positions? This is the exception rather than the common case.
Even the very best tech companies are constantly filtering through thousands of people who claim to have worked on extremely complicated development projects but can't code extremely simple things.
After a grueling interview process at Goldman Sachs, with 7+ technical interviews that required me to solve very specific questions on college-level Maths, Stats and Computer Science (admittedly, I was applying for a quant job position), I was eventually asked to interview candidates myself. While I did not feel entitled to change the current interviewing culture at the company by asking questions of a completely different nature from what I got asked in the first place, I conducted my interviews in a very similar way to what you described. By no means did I expect applicants to reach a definitive answer, but I instead worked on the problems with them to see how far their intelligence, creativity, curiosity and, most importantly, their ability to well communicate their thought process would take them.
Such interviews used to take a while hour of my time (which is a lot to afford when you work in a bank), but by the end of each I believed to confidently ascertain the candidate’s ability to thrive on our team. In retrospect, it has served me really well.
All in all, the problems posed (and the solutions given to them) might not carry as much weight in the final decision as the discussion held with the applicants. As long as the questions asked give them some material to work on, and DS and algorithms usually serve this purpose very well for us engineers and developers in general, one should be able to effectively select candidates given some time investment.
Isn’t the nfl scouting combine bullshit though? The players who tend to be most favoured are those who did well at playing football in the previous season, rather than those who scored well on the IQ or creative writing or jumping tests
- General IQ. Can this person understand and apply complex ideas
- Grit. Is this person hard-working enough to learn things that take time and effort
It's the software equivalent of the NFL scouting combine. The goal is not to create a test that is similar to the day-to-day job. But rather, create a test that isolates and evaluates a specific set of skills, which you think are important to the organization.