I asked him to explain how to do it in 2. And he was like, “what? It’s easy!” Then another person on the team who over heard it also asked.
Eventually the whole team was there, asking him to prove it, and he spent over an half an hour trying to explain it, and none of us were getting it. The consensus from everyone (except the manager) that it was an unfair question that relied on a a very subtle assumption about the data that wasn’t obvious at all, and that 3 passes was the minimum required Without the that assumption.
Of course the manager said everyone was stupid and it was a good question.
That manager remains my counter example of what a good manager is.
A friend of mine at a FAANG recently dealt with a bad hire. Their guess was that in the 6 months the bad hire was there they cost the company maybe ~5 million between wasted engineer time and delayed release schedules after people kept having to put out their fires. On the other end of companies, I've heard of seniors corrupting the database and its backup and ending an entire startup.
This is not a defense of whiteboard interviews per se, just an observation on why companies desperately want to avoid bad hires.
I've worked with one or two of those, but I've also been on interview loops that rejected candidates who went on to massively successful careers at a different FAANG.
The cost of false positives is relatively easy to quantify, the cost of false negatives significantly harder. Which is higher?
I'm always curious how people interview doctors. As bad as getting a bad programmer might potentially be, getting a bad doctor must be exponentially worse, so you'd think they'd have vetting the unqualified folks down to a science by now.
Mistakes can also be career-ending and potentially result in criminal prosecution.
It's way harder to get hired as a doctor than "do a bit of leetcode on a whiteboard".
I'd imagine software/IT might be only place where people who have delivered a couple of toy sized, half-assed webapps are now experts commenting on 'engineering challenges' and 'industry trends' with profundity.
They go through several years of medical residency under supervision before becoming a full fledged doctor and are burdened by expensive liability insurance and ongoing education requirements. That takes care of the competency beforehand.
It's not material to my point either way. I was only arguing that for some company's being very afraid of bad hires may be sensible and dismissing their concerns as hysteria does not seem obviously correct to me.
I think he was trying to point out that if some of the bad candidates had passed a whiteboard test, it brings in to the questions of the effectiveness of whiteboard tests to filter out bad candidates.
I remember a guy we hired years ago who answered all the tech questions we asked with flying colors. He was lazy and unmotivated and ended up dumping all his work on other people (me). I would have much rather had a motivated person who I had to teach some stuff to rather than him.
Who said that?
Others in the comments have expressed some legal concerns about "discussions" instead of Interviews, but I feel this is a hyperbolized fear based off a laypersons interpretation of US hiring law. The law requires you ask the same set of questions of all candidates, yes, but it reasonably understands that questions on past projects/work of course will never be the same, and these are legal and valid areas to go off-script.
So suppose in your situation you have a candidate and they talk about the project they worked on. To hear about their contribution, you ask (as you would all candidates):
"Tell me about a problem you encountered on this project that was particularly challenging from a technical perspective"
The candidate will tell you and likely their solution, and then you can start with the why's and how's.
"Why did you choose this approach? What are the benefits?"
"How did you arrive at this conclusion?" (you can further restate the question with "explain how you got to the idea that you needed to get here")
Make it a hard rule for yourself that you must be able to build the entire timeline and understand the candidate's role in that timeline. I find that even if it turns out they mostly were receiving orders from a more senior resource but could reasonably explain in their own words now why they ended up doing what they did, this is still a good candidate for me as they used available resources and took away a good lesson. If there is ambiguity, you can even just ask "why do you think your senior colleague had the right idea here?"
The idea of a discussion versus a checklist is you want to learn how a person thinks and how they approach issues. Even as a follow up to more factual check-list questions, "why" is very important as it helps you to understand the current state of the person. Just be honest with yourself that not everyone will have the same background and be honest on whether the "why" is relevant for the position. For example, some trivial linux kernel knowledge I would consider "nice to know", but it's by far not essential unless they're doing specific dev work on that field or claim to know it.
The other big catch you need to train yourself for is accepting that people will do things far differently than you do, even things that you consider wrong, but they work. The important part to focus on isn't about how close their answer is to yours, but to make sure you understand how and why someone reached that conclusion.
Why and how are very safe and legal questions as a follow up to a very straight-forward technical question.
Does it? I have never heard of such a requirement in a law (though I’m not a lawyer). That might be an employer’s decision on policy they use to ensure compliance with the law, but I couldn’t quickly find any law requiring the questions to be uniform across candidates. (It’s also a difficult set of terms to quickly and confidently exclude the possibility that one exists.)
I guess that’s a longer (and hopefully more polite) way of saying “citation, please.”
https://www.eeoc.gov/employers/small-business/3-im-recruitin...
Screen applications consistently. Apply the same standards to everyone applying for the same position.
Effectively, you should have the exact same criterion for all candidates for the same position and avoid "on the fly" questions/too heavily customizing it.
The legal discussion on all of this has basically boiled down to that you must ensure you're going through the same process for each candidate with the expected variances based on their employment/personal history (as it's relevant to the position)
I may overstate the situation slightly with that statement, but the idea is that if you ask about a networking stack for candidate A, you should ask the same probing questions for candidate B also. (Consider maybe you know that candidate B doesn't know this stack and will look "negative" on a review to the hiring managers, but you really like candidate B for other reasons)
Ultimately the idea is less about the specific questions and more that Candidate A and B are both reasonably considered in the same fashion for all of the requirements of the position, and that you aren't adding/omitting elements that may influence the applicability of the candidate in any fashion.
So let me restate: It's not required you ask verbatim the same words for each candidate (this is the lay-person over-read).
But for a given position, it should be established in advance a series of competencies that are necessary for the position and that are to be asked of each candidate. How you go about investigating it may vary from candidate to candidate, but should two interviews be reviewed, it should be expected that both interviews cover the same topics to a reasonable degree. (e.g., it is reasonable that if you ask "have you ever worked with library X" and the candidate flat out says they cannot answer questions on library X, you have reasonably covered that topic with this candidate. If you ask another candidate and they do have experience with library X and you ask some additional questions, you have not violated the spirit or letter of the requirement in this case.)
But, if you wanted to ask the recruiting team to add an interview to get more feedback, this would be a clear no.
Some of the best candidates I’ve been involved with hiring have been compelling for reasons you’d never get to in a very structured way.
For instance, a candidate who was just out of university. They were very shy about technical topics, and their tech depth wasn’t great. They went to a university in Africa that I knew taught people more stuff that was wrong than right, and they were starting at a disadvantage. To get him out of his shell I asked about things he’d done outside of academics. He told me that he had not done much because he was very busy. Busy with what?
Oh, just starting a real estate investing side business with his family where he’d been responsible for their international expansion into the UK. With no finance background, he’d spent his time understanding the financial, legal and tax implications of their expansion and successfully launched in his third year of university.
And I’d done a bit of finance so we talked about concepts there and he was able to walk me through complex topics in simple terms.
My job for that interview was judging bias for action, curiosity, and diving deep into metrics. He couldn’t show that for tech because he went to a horrible university and had been very busy with other extra circulars.
He apologized to me for spending so much time talking about non tech subjects. And here is the problem: people who are bad at interviewing are punished by rigid processes. There’s a lot of data to show that women and immigrants from certain backgrounds are more reserved on average. You have to be a much better interviewer to pull off a structured interview that lets these candidates shine.
Stripe handled this by having an extra interview baked in by default. People who were viewed as no risk hires didn’t have to do it, but the default was an extra interview to cover risks. Because it wasn’t an aberration, it wasn’t giving a candidate preferential treatment.
I honestly think it’s one of the smartest interview loops I’ve seen. I think they are one of the few large companies gambling a bit on the false positive side of things to shave off false negative points. And honestly, I was impressed with the people I interacted with at stripe more than AWS. They’re doing something right.
Making definitive statements about liability is not trivial. I can confirm that at some very large companies legal has banned this style of interview citing discrimination concerns. I'm not a lawyer, so I can't comment on whether they're call is more or less correct but it certainly doesn't seem settled.
Also working for a "large" US company and discussing with peers from others, no such banning exists. There is very clear training and a heavy restriction on who can perform interviews, but this is not the same as outright banning it.
I do a coding task first and a Q&A afterwards — because these are the requirements.
I did many interviews. My experience is: I could skip the Q&A completely. Most candidates could answer the questions just fine after reading the Wikipedia article for 10 minutes. In the interview I can see if they already read it or not. But I don't think it matters.
What matters are the coding skills. The coding task is quite simple and half of the candidates (with master degrees and 'years of industry experience') fail. But these candidates are often good talkers when they are chatting about the benefits of agile methodology and so on.
I totally agree with you that a candidate should just write any program. I believe I know after 15 minutes if they are developers or not.
But I also agree with the idea that we interviews must calm down the interviewee and remove the nervousness. I don't want to see if they stay cool in an exam situation because daily work is not an exam situation. I want to see if they can code when they are relaxed.
If half the candidates with a master's degree in CS are failing your test, that should be a pretty huge red flag to you that your test might have an issue. What question are they failing?
C++ is a particularly badly affected language for some reason. I think there are a lot of people who learned C++ once, a long time ago, then moved to Java as quickly as they could when it came out. But nobody likes to admit that maybe they lost a skill they once had, so they keep putting it down on the CV regardless. And then when asked to write something like hello world, they don't remember how to use std::cout or whatever.
These guys are applying for C++ dev jobs but are completely unable to write a program. Most of these fake programmers have studied electronics (not CS) or have studied in a country where you can buy a degree.
What are you actually inferring from the situation - how well the candidate can write software? How well they tackle stress and anxiety? How much spare-time they have to grind these types of questions? etc.
Probably a mix of them all, but there's a lot of irrelevant noise - if you're just looking for one thing.
As companies ask 'Leetcode easy' questions, there came to be thousands of online blogs, youtube channels etc.. which trained even the n00bs who can't write good code otherwise. If they train very well they can solve 2sum or write binary tree level order traversal without understanding much. Of course not all interviews can use novel exclusive questions, and these have a non-negligible chance of passing.
Now if you are hiring, you would think, "If these n00bs can solve Leetcode easy with practice, the good ones are solving Leetcode medium with same level of practice. So let's raise the level of questions so that we don't end up hiring these rote-learning noobs". And it continues.
I don't think there's an obvious solution to this, if you don't want to lose the statistically good heuristic of problem solving skills, in order to weed out candidates.
Everyone you talk with, has probably one or two anecdotal stories of such. "Yeah I worked with this CS grad that couldn't even write FizzBuzz" - yet we ignore the hundreds of other that do their work just fine.
And this is fought with setting up ridiculous 8-part technical interviews where you'll have to whiteboard some leetcode questions ("Given a problem, show us a working O(Log N), or preferably O(1) solution. You have 45 minutes") or design questions ("Show us how you would design slack / discord / zoom / etc.") that drags over 6 months.
For someone to be truly incompetent, or even not good enough to meet the company standards, the current system is overkill.