Strongest possible disagree. There are people who can talk all the way down to the point where they're basically dictating code to you, who nevertheless cannot deliver on a real project. There's a difference between knowing, intellectually, how to solve a problem with code, and actually being able to deliver working code.

In practice, though, what really seems to happen is that technical interviews are split between whiteboard-grade programming puzzles and "friendly deep discussions" about technical problems that both (a) can be prepped for with rote memorization and (b) are extremely game-able by people with strong interviewing skills (for instance: the skill to respond to questions with questions, or to redirect the course of a line of questioning back to safe ground for the candidate).

It's not like we were asking candidates "hey, how did you enjoy your last software security job?". We were doing something closer to "here's a piece of code with a heap overflow in it; talk us through how you'd write an exploit". It didn't work: there were people who could answer those questions indistinguishably from an exploit developer who nevertheless couldn't write a 1995-era stack overflow exploit if their life depended on it.

> It's not like we were asking candidates "hey, how did you enjoy your last software security job?". We were doing something closer to "here's a piece of code with a heap overflow in it; talk us through how you'd write an exploit". It didn't work: there were people who could answer those questions indistinguishably from an exploit developer who nevertheless couldn't write a 1995-era stack overflow exploit if their life depended on it.

Something about this example is not registering for me. I can't imagine how this situation is possible. Going from a detailed, step-by-step description to a code implementation is a trivial rote task in my mind. I must be missing something.

I mean, I've seen very technically proficient devs fail to deliver because there's too many cooks in the kitchen, or because they drown themselves in tooling or bikeshed their project to death. Maybe something like that happened?

There's definitely tons of senior devs out there that care way more about linting and unit testing than they do about writing the code that implements the spec.

Nothing hit quite as hard as the punch to the gut from the bit about linting and unit testing... spot on. I've already decided to move on and such, but my current company is so this that it hurts.

I don't have a perfect theory for it! It's just something I've repeatedly seen happen.

Happened to me. I hired someone onto my team who did great during the interview. The questions weren't puzzles, but live coding sessions. He got all the way to the point at which he had written a semi-functional web server.

This guy couldn't deliver a single project and would spend days debugging issues before he asked for help and I or someone else on the team would solve it in a couple of hours or less. I have no idea why this happens, but I've seen it twice.

> Strongest possible disagree. There are people who can talk all the way down to the point where they're basically dictating code to you, who nevertheless cannot deliver on a real project. There's a difference between knowing, intellectually, how to solve a problem with code, and actually being able to deliver working code.

I'm a little confused by this statement. Is your issue that they aren't _capable_ of writing the code or that their job performance isn't there once hired? i.e. Are the technical chops not there or is the work ethic not there? It seems like you are describing the later to me, or we are talking about different levels of conversation.

Being able to talk intelligently, in depth, and at length about code is not the same skill as being able to write or debug code.

I haven't experienced this, not if you are digging into detail on how they solved previous issues. I'm not saying it doesn't exist, but I've never dug into detail and had coding be the problem. If you leave the conversation at a conceptual level, then I can see it happening much more often.

I'm not OP, but when I've experienced this, it's very similar to something like "I've seen a million heist movies, but absolutely could not steal shampoo from a CVS". There's a very big difference between talking about doing something and actually doing it; there's a level of executive function that's required.

More specifically, you make so many small executive decisions when coding. Naming, how functions and classes are designed, which file/module this goes in, is this already tested, oh I found an unrelated bug, do I make a ticket, a new PR, or include it in this one, etc etc etc. The only thing I've seen that can answer those questions is homework--maybe previous work, but homework is better.

But I think you're remiss if you're not asking your senior engineering candidates how they make those decisions, not because the answers to relatively trivial questions are important, but because you should know--and they should be able to articulate--how they make those decisions, and they should know the pros and cons of their approach and other approaches.

> It didn't work: there were people who could answer those questions indistinguishably from an exploit developer who nevertheless couldn't write a 1995-era stack overflow exploit if their life depended on it.

Do you think the problem was that they couldn't write that particular kind of code, or was it the more general problem of not being able to code well in general?

If you've never been around real bullshit artists it's hard to imagine, but there are people who can talk the talk so well and say all the right things, and yet their actual programming is terrible.

There are also the opposite. Folks who are not great at verbalizing programming principles, but do deliver solid code in reality.

In summary, hiring is hell...

Bullshit artists are honestly easy to spot. They are easy to trip up because they think they are good at fooling people. If they said all the right things, the questions were probably low quality. I would throw a zinger in to see if they can react to it. I honestly expect a candidate to say, "I am not familiar" or "I'm not sure I'd have to look that up" once during an interview.

Well... There are different kinds.

The people I talk about aren't even deliberately fake. They really think they're awesome. They can say all the right things. They're charismatic and sound real smart. They have great anecdotes, dripping with wisdom.

It's just that their work is mediocre at best.

> The people I talk about aren't even deliberately fake. They really think they're awesome.

I guess you don't have to be good at fooling people when you believe it yourself.

(cf. Theranos)

What happens to every candidate who gets flustered by the questioning and seems to bow out earlier than their technical depth?

How do you hire people who are more technical in a subject area than your current staff, if your test bottoms out at the depth of the current interviewer in the room?

> What happens to every candidate who gets flustered by the questioning and seems to bow out earlier than their technical depth?

I'm not sure I get this question - I'm not talking about a white-boarding exercise. I'm just talking to the candidate about a technical project they have worked on in the past, the pain points, how they resolved them, etc. If that results in the candidate being flustered it's not a good sign on numerous axes.

> How do you hire people who are more technical in a subject area than your current staff, if your test bottoms out at the depth of the current interviewer in the room?

That is a _very_ different hiring situation. If you have no one in the company that is equipped to properly technically interview the candidate they obviously you can not properly technically interview the candidate. You'd likely have to leverage the connections of the management team to find the right person to lead up unknown territory.

One of the very best people we ever hired was so flustered in his final interview with us (a formality, after work-sample challenges) that he was visibly shaking during it. If we had taken the "numerous axes" on which the was "not a good sign" seriously, we'd have missed that hire --- and, I think, an entire business unit in our firm wouldn't have been started.

I generally think software developers (I count myself among them) know far, far less about psychological assessment than they think they do.

Of course that happens, but you're taking a real risk in hiring someone who can't get through the interview. I'd rather miss out on a good hire than make a bad one. You only have so many ways to determine whether or not someone is competent.

I don't know what to tell you. We switched to pure work-sample hiring and hired several dozen people that way. We retained all of them. I can't say that about our interview process prior to that: not only were we not picking up the "moneyball" candidates I'm talking about here, but we were also hiring people we (painfully) ended up having to let go.

The only time people face the specific pressure of an interview is in an interview. I promise you, whatever generalization you're making doesn't hold.

My point is that you can't easily discern why they are performing poorly. All things being equal I'll take the person who could answer the questions.

I have an anxiety disorder, so in depth conversation with a stranger is going to cause me to be flustered. If you asked me to give the answer in writing I could. If you asked me to code on a whiteboard I could. If you asked me to do a takehome I could. But a verbal conversation with a stranger judging me in an interview setting is the worst out of all possible options for me.

100x this. Whiteboarding is so outside the typical engineering process that it just feels cruel, especially to people with any kind of anxiety. It's like if you were hiring construction workers and you asked them to perform an interpretive dance about concrete in the interview. It's just not at all the job and very difficult for many of the people who go into programming in the first place. It's hard for me to think of it as anything other than hazing.

> If that results in the candidate being flustered it's not a good sign on numerous axes.

Are you hiring sales reps or engineers? Why are you asking for an engineer to sell you on their past project(s)?

Interviews by their nature are about judgment, and knowing you're being judged by a complete stranger who holds the power over this potential path of your future is enough to get anybody flustered, especially a brainiac that just wants to work in relative peace behind their desk.

> Are you hiring sales reps or engineers? Why are you asking for an engineer to sell you on their past project(s)?

I don't ask them to sell me, I ask them to dive into the technical issues they faced and how they solved them. It tells me a lot more about how they will handle similar issues in the future than scribbling DFS on a white-board would.

Just to clarify, when you say "how they would handle similar issues" do you mean talking to a hiring manager about a past project during an interview setting?

I don't understand what you're calibrating for...

I'm a bit unclear on your question. I'm assuming a technical person with relevant knowledge performs the technical interview, if that is what you mean.

The interview setting, with someone who might or might not be technically capable, with your job / career / earnings at stake is a very different situation than explaining past projects to a colleague at work.

Programmers in a company work with other programmers. If they cannot explain some work they've done then they are useless. Or would you hire a programmer who does not speak English?

I think what the OP is hitting is the difference between being able to explain something and being good at it.

With a deep enough interview process you’ll find people who know what they are doing. But it won’t tell you anything about their practical coding skills; it won’t matter if you are hiring an architect, but if it’s for a main developer you’ll want a decent level of productivity, proficiency with the tools, application of best practices and reviewing skills.

Those aspects are a lot easier to understand with actual code produced.

There have been instances where I have forgotten vocabulary that makes me seem knowledgeable about a project, but I have nevertheless done the work. I sound incompetent but I actually am competent.