The thing is: The method described in the post has the same problem. It just doesn't look like it. Most of these types of interview templates are designed by one or two persons and then blindly applied. Sometimes you have a view modifications, but for the most part it's fixed.

Since every interviewee gets the same set people then start to think this is more scientific than "gut feeling" - it is not, but instead of your own biases you have the biases of the template designer.

There are probably a few companies which extensively study if their interview process for the people hired (the only subset they have access to, which already is a problem) actually matches their job performance and then try to readjust the process, but it doesn't seem to happen very often.

This also gets us back to the old problem that we have no idea what a good programmer is or how to evaluate job performance.

I find that asking "What is the worst bug you've ever encountered" is a great sounding board.

I think that question can be great, but it also puts a lot of pressure on the candidate to execute a rather complicated query on the spot. I've definitely encountered lots of interesting bugs in my time, but my initial reaction to this question is a total mind blank. Especially since "worst" could mean a lot of things, and what I really want is "worst bug I've encountered that I can say something interesting and impressive about".

I'd say it's not a bad idea to forward questions like that to the candidate in advance of the interview. That way, you avoid false negatives from people who just couldn't think of a good answer under interview pressure.

Would “what is one of the most interesting bugs you’ve ever encountered?” allay your concerns? I think that’s the main intent of the question above.

(I wrote the article)

I never ask this sort of question because:

1. I'd struggle to answer it myself despite nearly three decades of fixing bugs. I could probably come up with some interesting ones after a few minutes of thinking, but most of the bugs that'd immediately pop to mind would just be recent bugs that were hard to explain without knowledge of the codebase.

2. There's no way to guide an interviewer on how to mark this question beyond gut feel. What exactly does a good answer to this look like? More importantly: what does a bad answer look like? Asking questions which are impossible to actually fail at, or for which the outcome is basically random and unjustifiable, is a very common failure mode for interviewers.

3. It tells you nothing about the candidate because they can easily just repeat a bug they read about on a blog last week. You can't check they actually debugged it themselves.

For instance I just searched for "most interesting software bug" and clicked the first relevant link, the second story on this Quora post could easily just be repeated verbatim to an interviewer over the phone:

https://www.quora.com/What-is-the-most-interesting-software-...

If the interviewer wasn't alert they might be fooled by it. Candidates will quietly Google for answers during talking questions whilst trying to avoid the interviewer noticing, and many other things.

That's why I emphasise that designing good interview questions is quite hard.

(I wrote the top comment)

I agree that's not a great question. I usually have few simple questions that can tell you if your candidate really understands the platform they say they understand -- it doesn't take much.

I usually have one or two projects set aside specifically for new hires; the type of project that is maybe a month-long or longer but that doesn't really require any domain knowledge or big integrations. It allows them to have something to do when they're learning everything else. In the interview, I can ask them how they'd proceed with it and I always offer that I'm here to help them and will answer any questions they have. This is literally no different than what I would ask them on day one of the job if they got hired.

Is that more informative than whether or not they can reverse a linked list? I think so.

> Would “what is one of the most interesting bugs you’ve ever encountered?” allay your concerns? I think that’s the main intent of the question above.

I prefer the even more casual, "What's an interesting bug you've been bitten by?"

I think I like this variation the best, because it opens things up to bugs in tools that one has used, and other kinds of bugs that aren’t necessarily ones the candidate would have had to attempt to fix.

I think that's a better way of phrasing the question. However, I think my real issue is just that it's difficult to rummage through your mental database of past bugs in the space of a few seconds. (Realistically, it's awkward to pause for more than a few seconds before answering a question in a face-to-face interview.)

What would be the downside to giving the candidate a heads-up that you're going to ask this question?

Maybe, but this is something I'm very bad at. I forget about problems once they're solved.