Now if he is saying, "well they said python and I hate python, so they are wrong!" Then, obviously, that is not a good interview tactic. But I didn't find that to be the case from the way it was presented in the article.

Those are the big theoretical problems with questions like that. But the pragmatic problems with it are even worse: they are a lens that focuses and applies subjective biases on the part of interviewers. Developers are terrible psychologists, and subjective interviews are the one time that we demand that software developers perform as psychologists.

The best job interview questions are strictly and (as closely as possible) objectively predictive, and the easiest predictive questions to ask are work-sample questions: "here is a problem we solve routinely in this job, candidate, please solve it yourself."

You should not be apples-apples comparing candidates anyway. Everyone is different, and it is a collection of skills that you need to be a developer.

Candidate A may be a person who isn't as quick with coding, but they can quickly architect a solution in their mind and formulate an opinionated argument about why it is superior.

Candidate B may be an incredible coder, but unable to explain their methodology, or architect a solution that is generic and re-usable.

Both of these candidates can be an asset to the team in different ways. I find a lot of developers, the "JUST LET ME CODE" types, really don't understand the communication skills needed to develop an application with a large group of people. This question helps find people that will be better in that group situation, regardless of how strong their coding skills may be...because coding is only a portion of the job, in the real world.

Edit: I also pointed out in my reply, that an interviewer who takes a subjective bias when assessing the answer to this question is doing it wrong.

In the 12 years of my career prior to discovering objective, work-sample, standardized, data-driven hiring, I hired exactly the way you're advocating. I sought out questions like the one you're advocating for. My favorite question from that time: "if you had to put together a libyou.a that you take from project to project, what would be in that library?" I found it to be a great way to start a conversation with a candidate to get inside of their head, and also liked the fact that it quickly gave me a signal as to whether the candidate had actually done the kind of work we did in our shop.

As it turns out, the ability to project a signal as to whether a candidate has actually done the kind of work we do in our firm is not actually useful. What's useful is: "if the candidate is put in a reasonable position to try, can they actually perform the tasks the role requires". Surprisingly --- in fact, shockingly --- I've found zero correlation between the capacity to signal ability and the capacity to demonstrate ability. ZERO. Our best performers six months in, as often as not, interview terribly on "signalling" terms, and were hired because work-sample testing overrode those concerns.

Opinionated argument about half through unprepared ideas are fun, but waste of time when done during meetings.

A lot of development shops work by picking the best technology for the job. They choose the best plugin, or the best language, or the best component. In those types of shops, this question is exactly as you say - a task they will have to do routinely in the job.

I think work-sample questions are really good, but you're still optimizing for people who can solve problems in the extremely artificial context of a job interview.

When you think about it, it's odd that most programmers aren't interviewed this way. As a firm, the most important question I want answered is "Can the candidate do the job?" There are obviously other important considerations, but that's numero uno. And the best way to tell whether the candidate can do the job is to ask them for a small sample to prove it.

I'm 100% certain I've been passed over for positions at other places because the hiring process isn't generally designed to answer that question accurately. It's something I want to keep in mind for when I'm on the other side of the table.

"My favorite language is Haskell because it lets me reason about the correctness of my program before I even try running it."

"My favorite language is Python because that's what we used in school."

"All I know is Matlab, and whenever I've needed to program something I've always been able to do it in Matlab, so I never bothered to learn anything else."

"I guess my favorite language is C#. That's what pays the bills. Everything at work has always been written in C# or Visual Basic."

"My favorite language is Java, because that's what I use to write Minecraft mods, but I actually work as a COBOL programmer where I have twenty-five years of experience."

All of these answers are technically "correct", but the interviewer will inevitably judge some of them to be "more correct" than others. It's subconscious and it can't be helped.

Moreover, when you get together with a team for a post-mortem and get asked what you think about the candidate, the answer you give will be culturally conditioned. So I can predict what many teams are likely to say: The Python person is "awfully inexperienced". The Matlab person is "a bit narrow". The C# person is a tool, of course, unless we actually work with the Microsoft stack. The top marks are likely to go to the Haskell or the COBOL person; victory may hinge on the team's prevailing opinion of Minecraft.

Which of these answers is most indicative of potential success as, say, a Rails developer? None of them. There is probably no signal here at all.

There's no wrong answer, but if your resume says you've developed projects in C and in Java but you are not able to articulate any reasons you might pick one over the other, I'm a little suspicious. It doesn't even really have to be languages; I mostly just want to hear you have a well-reasoned opinion on something.

Is that a well-reasoned opinion? "I'd like to pay the rent this month" is a good reason. "The company doesn't have the time to reimplement working software on a different platform" is a good reason.

Of course, a well-socialized programmer understands that these answers, while truthful, are insufficient, and will have duly memorized a stock answer like "You pick C when you really need runtime performance and don't mind all the segmentation faults, otherwise you pick Java."

If you gave me that "stock answer" I'd ask followups. This isn't a trivia question with a secret "right answer" password.

And hey, it's not the Worlds Greatest Interview Question, but I like it a whole lot better than "explain Quicksort" or "write out pseudocode for calculating a checksum on the whiteboard"

At any rate, I would vastly prefer it to "where do you see yourself in five years?"

I am saying that as someone who would confidently blable about agile development (it is the thing back then) and architectural designs/patterns without experiencing them on a real project at that age.

Those questions may show whether the candidate has general well roundness, but they are not likely to distinguish those who can do from those who can not do. Someone who focus on solving problems and coding will be able to answer them to some level, but it is very easy for him to answer less well and in less details.

That's another problem with subjective interviews: it allows developer interviewers to apply their biases about the genealogy of knowledge to candidates.

By that I meant that if interviewing a candidate you should be able to say w/ reasonable accuracy if they can a) actually write code (personally I don't usually care if it involves the tech stack I'm using) and b) have a strong breadth of knowledge on different paradigms, tools, languages, etc so that their decision making will be well informed going forward. Note that my lack of caring about tech stacks/languages/etc is because I've found people who are strong at b) are able to pick those things up rapidly, assuming that they actually can code in the general sense.

Absolutely.

I think several of the responses to this post are missing the key word here - subjective. Asking subjective questions and qualifying them with subjective criteria like 'reasonable' is necessarily going to allow for the subjective beliefs and opinions of the developer turned 'CEO for an interview' to affect the business.

Have you seen people who mesh perfectly well but go to hell in a handbasket when you throw them in an interview? It feels like that is something, however arbitrary and difficult to test, that you'd catch when you interview these people - avoid anyone whose fitment is questionable at all, and that your statement is merely a byproduct of not catching these bad interviewers before you extended them an offer.

My thesis is that practically all software developers are bad interviewers. I'm not suggesting that there are a couple bad apples that turn into tyrants during interviewers, but rather that the interview process is a vector that pushes software developers into a subtly bad place, in a Stanley Milgram sort of way.