> It's not like we were asking candidates "hey, how did you enjoy your last software security job?". We were doing something closer to "here's a piece of code with a heap overflow in it; talk us through how you'd write an exploit". It didn't work: there were people who could answer those questions indistinguishably from an exploit developer who nevertheless couldn't write a 1995-era stack overflow exploit if their life depended on it.

Something about this example is not registering for me. I can't imagine how this situation is possible. Going from a detailed, step-by-step description to a code implementation is a trivial rote task in my mind. I must be missing something.

I mean, I've seen very technically proficient devs fail to deliver because there's too many cooks in the kitchen, or because they drown themselves in tooling or bikeshed their project to death. Maybe something like that happened?

There's definitely tons of senior devs out there that care way more about linting and unit testing than they do about writing the code that implements the spec.

Nothing hit quite as hard as the punch to the gut from the bit about linting and unit testing... spot on. I've already decided to move on and such, but my current company is so this that it hurts.

I don't have a perfect theory for it! It's just something I've repeatedly seen happen.

Happened to me. I hired someone onto my team who did great during the interview. The questions weren't puzzles, but live coding sessions. He got all the way to the point at which he had written a semi-functional web server.

This guy couldn't deliver a single project and would spend days debugging issues before he asked for help and I or someone else on the team would solve it in a couple of hours or less. I have no idea why this happens, but I've seen it twice.

> Strongest possible disagree. There are people who can talk all the way down to the point where they're basically dictating code to you, who nevertheless cannot deliver on a real project. There's a difference between knowing, intellectually, how to solve a problem with code, and actually being able to deliver working code.

I'm a little confused by this statement. Is your issue that they aren't _capable_ of writing the code or that their job performance isn't there once hired? i.e. Are the technical chops not there or is the work ethic not there? It seems like you are describing the later to me, or we are talking about different levels of conversation.

Being able to talk intelligently, in depth, and at length about code is not the same skill as being able to write or debug code.

I haven't experienced this, not if you are digging into detail on how they solved previous issues. I'm not saying it doesn't exist, but I've never dug into detail and had coding be the problem. If you leave the conversation at a conceptual level, then I can see it happening much more often.

I'm not OP, but when I've experienced this, it's very similar to something like "I've seen a million heist movies, but absolutely could not steal shampoo from a CVS". There's a very big difference between talking about doing something and actually doing it; there's a level of executive function that's required.

More specifically, you make so many small executive decisions when coding. Naming, how functions and classes are designed, which file/module this goes in, is this already tested, oh I found an unrelated bug, do I make a ticket, a new PR, or include it in this one, etc etc etc. The only thing I've seen that can answer those questions is homework--maybe previous work, but homework is better.

But I think you're remiss if you're not asking your senior engineering candidates how they make those decisions, not because the answers to relatively trivial questions are important, but because you should know--and they should be able to articulate--how they make those decisions, and they should know the pros and cons of their approach and other approaches.

> It didn't work: there were people who could answer those questions indistinguishably from an exploit developer who nevertheless couldn't write a 1995-era stack overflow exploit if their life depended on it.

Do you think the problem was that they couldn't write that particular kind of code, or was it the more general problem of not being able to code well in general?