It would be unreasonable for me to pass them through an interview that the other members on the team passed through based on "oh, they had a bad day".
Since you haven't been on the other side of the table, have you had coworkers who obviously were at a position above their skill level? Who allowed the rest of their team to carry them?
I've had periods wherey personal life has affected my productivity. That doesn't preclude my ability to write an algorithm that reverses a string nor write a simple SQL join. I don't ask brain teasers. I ask questions related to what the engineers under me will be expected to solve every day.
Also, are your people actually reversing strings by hand every day? I sure hope not!
My policy is to ask about those things instead. The ones you'll actually do rather than the ones that are, at best, tangentially related.
If you can't do that then you should probably worry about your own skills before testing somebody else's.
I lopped off a block of the code, made it run without the rest of the code base and explained what it was supposed to do. This wasn't that simple, but could still be explained in about a minute to somebody with familiarity in ML (assumed; that's what we're hiring for). I then challenged the candidate to spot any bugs and implement a feature.
This really wasn't that hard. I've done the same thing twice before in two completely different industries.
Why would that be impossible for you?
As I read it the main observable difference between your exercise and the kind of exercise you're arguing against is that you are starting with some baseline code that the candidate is supposed to modify. That seems like a pretty good idea, but I haven't used it yet.
I've interviewed well over 300 people, for positions from junior/entry-level to principal. If you read many of the comments to this article you'll hear other people with experience interviewing who also call out that candidates can and do lie. I don't like it, I'd rather it not be the case. I constrain the majority of my questioning to what a candidate claims to know on their resume. Here's a tip... If you don't know it (and I'm not talking about bullshit trivia questions), then don't put it on your resume.
I'm joking. (sorta)
I'm glad to hear that you tailor to the resume, that's actually better than most and my favorite approach.
But if I had a dollar for every time someone walked out of an interview with a braggadocious claim like "they couldn't even do X, they must be lying" but then answer "no" to "did you ask them anything else?". I'd be rich.
Reversing a string is a bad example, it is dead easy. But it seems a lot of people have random substitutions for it of varying obscurity and difficulty.
Just because a candidate fails or falls short in one area doesn't mean they are cut, but it's a pretty large red flag if a candidate purports to know something, both on their resume and in person, yet can't answer some seemingly basic questions. If you state on your resume that you architected and implemented a system end to end, yet can't whiteboard the result and explain bottlenecks or failure conditions, I absolutely will assume you stretched the truth a bit. And if the candidate feels the need to do that to get the job, what will it be like to work with them? Could you trust their estimates? Would you be able to back them if something they delivered fails and you trusted them as a senior to deliver?
I ask follow-ups. I give hints. Honestly I'm dismayed at the skillset of purported "seniors" and "architects", and that's interviewing for small companies up to one of the FAANGs.
Coding: on a whiteboard or with someone watching over you has no correlation with performance on the job.
Data structures: questions usually end up so contrite that they have no bearing to real-world situations. That or based on luck of seeing an remembering the right one. Data structures usually go hand in hand with algorithms.
Algorithms: questions become more like trivia or reinventing something that has likely got research papers on it. Oh, you want me to invent an algorithm I've never heard of on a whiteboard with your clues?
System design and data modelling: the kind of thing you can do in an interview lacks real-world data. No empiricism. It's all about the trade-offs.
Tailoring to the candidate introduces inconsistency and unfairness/luck.
What if these "seniors" and "architects" are frauds?! How else could they hold down a job for 20 years and not be able to answer my questions? They must be really bad!
Lucky my interview process is really great and I always hire the good ones. I track every rejection and I have proof that they never amount to anything!
Thoughts?
I agree that we don't code in a vacuum. What I'm trying to ascertain in the interview is how someone thinks, how they approach a problem and work through the solution. The more they communicate the better, even if part of that communication is "is there a length property on the String class?"
I note, but don't give it too much weight, if there are minor syntax errors. I ask candidates to walk through their code with an example input, especially the ones that have errors. If they catch their errors and fix them, I note that. If they skip over their errors because they assume the code does what they wanted it to, I note that.
Ultimately, can you reason through the problem, come up with a solution, and talk about the tradeoffs you made in your implementation. Any senior should be able to handle that without any preparation.
I worked at a FAANG where in several loop debriefs I was challenged on the complexity of my question. One of the challengers asks a chutes and ladders question (given a random chutes and ladders board, where you can choose between 1-6 for every move, what is the minimum number of moves to get to the end).
It doesn't take a hard technical question to figure out whether someone knows how to think and work through problems.
That's unfortunate, but logically inescapable.