What are the "all things equal" here, though? I would tend to disagree that someone who can complete the coding challenge faster is the objectively better candidate. Especially if we are talking about leet coding, where there are people who study their asses off and memorize the common leet code problems. Or maybe the coding challenge isn't leet code, but just so happens to be something that the candidate has done before, or has done recently. Just because one candidate lucked out and has done this very specific thing, doesn't mean they are better than a candidate who could figure it out in half a day, if given a little more time to do so.

You people are so bent on arguing the unarguable that the arguments are so unnecessarily argumentative.

The interviewer and the hiring company does not have a God's eye view of the candidate and must work within the limitations of the hiring process. The best one can do (without resorting to metaphysical processes) is to make a decision that has the highest expected value (or highest chance of hiring a good candidate).

The fact that there could be a candidate that lucked out because they were prepared for the specific set of questions you asked is not specific to live coding problems. You could have asked them anything and they could have lucked out and just happened to be prepared for the answer you were expecting.

That said, I can tell you for certain that if given a task that the average candidate takes 10 minutes, and a candidate completes it in half a day, you're looking at a 0.1x developer right there. That's by definition.

Normal interviews are mostly a scripted performance where people largely regurgitate answers to predictable questions.

So the issue isn’t that it’s a stand up performance it’s that you need to split your attention between the performance and solving some trivial problem. Becoming really good at programming requires being able to focus on an actually difficult problem not work through a simplified example while entertaining an audience.

There might be a correlation between solving trivial problems quickly and being able to figure out a heisenbug from some multithreaded monstrosity, but it’s not strong enough for minor differences in performance to be meaningful.

> Normal interviews are mostly a scripted performance where people largely regurgitate answers to predictable questions.

Ah, so you prefer being asked "where do you see yourself in 5 years?", and regurgitating the text generated by ChatGPT instead?

Alternatively, as I replied in the sibling thread, do you prefer to be handed a difficult take home task that is expected to take 2 weeks to complete?

I think I can live with the first, but I'd be insane to work 80 hours on a task just for a small chance to get a job.

“Where do you see yourself in 5 years?” is asking a perfectly reasonable question about what if anything you’re working towards. If you think regurgitating something from ChatGPT is appropriate, I can see why you might prefer more limited interviews.

> Becoming really good at programming requires being able to focus on an actually difficult problem not work through a simplified example while entertaining an audience

There are multiple ways to "become really good at programming".

A company of any size needs people who can churn through well-defined problems quickly, and most live coding tests are relatively well-suited to selecting for that. They also need people who do other things well:

* tackle large, ill-defined problems, alone or in a small team * identify, triage, and (if necessary) solve problems as they emerge * refactor existing implementations that have outgrown their initial architecture * identify trends and estimate when they will lead to scaling issues in the future * communicate complex problems to people without the necessary context * break down complex solutions into discrete, well-defined steps that can be tracked to completion

They also need people who can accurately estimate how long all of the above will take... unfortunately, as we all know, such people do not exist.

If you're hiring someone to churn through well-defined problems, live coding interviews are likely a good fit out of the box. If you're hiring someone to reinforce a weakness in your organization's ability to do any of the other things above, live coding exercises are at best a loose framework that we're all familiar with to try to get at whether or not they have those skills.

I'd even argue it's a prerequisite.

If one doesn't even know how to complete well-defined tasks reasonably quickly, I can't imagine how they would be able to break down complex tasks to these smaller, well-defined tasks, and accurately estimate the expected effort needed.

This sounds almost Dilbert-esque. It seems like a lot of people here want to be a pointy-haired boss (or pointy-haired tech lead)!

Coming up with your own tasks means understanding the problem implicitly. Solving other peoples trivial problems means searching for the interviewer’s hidden gotchas and unspoken requirements.

Ie if someone asks you to find the median value of an array the number of elements and how to handle nulls isn’t obvious, but when you want the median value in an array the context is clear.

> I'd even argue it's a prerequisite.

You can know how to do something without being able to consistently do it.

Maintaining velocity is something that I've never been very good at. I always struggled as a junior and mid-level developer with having periods of very high productivity followed by periods of very low. I've been fired for it.

On the other hand, if you measure my output on a monthly or quarter time horizon, I'm very consistently at the top of the class.

My strong suits are around helping others get through tough problems, quickly switching between levels of abstraction (including within a single conversation), understanding how a part of a process fits into the whole, and in guiding/limiting architectural discussions by managing the scope of the problem space under consideration.

This fits very well with my current "staff level" role, but was a real struggle getting to this point.

> This sounds almost Dilbert-esque. It seems like a lot of people here want to be a pointy-haired boss (or pointy-haired tech lead)!

I can see why you'd say that, but I think there's a big difference: PHB was incompetent in addition to having a different role from his team. I've found a sweet spot for myself where I can fill the parts of the "PHB role" well that actually do help the team, but I'm also more than capable of switching contexts and knocking out some tickets when there's a crunch. When I'm especially "in the flow" I work on that sort of stuff and do well. When I'm not, I run interference and keep my team from having to deal with the organizational stuff that doesn't directly help them get their jobs done.

The best functioning teams I've experienced all have one thing in common: they set people up for success by allowing them to do play to their strengths. As a team grows, you can find gaps in those strengths. I see hiring as an opportunity to find someone who is strong in the areas the team is weak. Let the new person come in and do the things they like to do and are good at!

They’re only objectively the strong candidate if that’s the sole thing the company values. It’s unusual for that to be the case.

With the "all things equal" qualifier, it's sufficient that completing the task faster is a positive signal rather than a negative one.

I mean, this is very elementary.

> then objectively the other candidate is the stronger one

...if the job is to complete abstract problems in live coding interviews, sure.

By your logic, there's no point in doing any interviews at all, because it is nobody's job exactly to sit as a candidate in an interview session.

Is that your point? Because I'm getting tired repeating the point about take home tasks in the sibling comments by now.

a single example of a one minute difference in solving a problem that takes ~10 minutes is noise

Yes, so given that all things are equal, are you going to hire the slower person then?

Or are you going to, as I mentioned in sibling comments, give them a take home task that takes two weeks to complete, just to be sure?

> All things equal, if you can complete the problem in 10 minutes and the company can hire somebody that can complete it in 9, then objectively the other candidate is the stronger one.

They are stronger in the specific problem space you’re testing for. If your business has a lot of novel coding problems that need to be solved in under 9 minutes, then sure, this is a great measurement.

On the other hand, if your business has a lot of hard problems that take days, weeks, or quarters to solve, measuring someone’s ability to solve a 9 minute problem is a terrible metric. If anything it’s a counter signal, since it tends to select for candidates that optimize short term thinking over long term planning and problem solving.

> if your business has a lot of hard problems that take days, weeks, or quarters to solve

I suppose your ideal interview as an interviewer would be to give the candidate a take home task and ask them to spend 2 weeks to work on it?

> If anything it’s a counter signal

Or perhaps give the candidate a task that normally takes 30 minutes, and hire the candidate if they take 60 mins to finish it?

I mean, you do you, but I hope you (and everyone else) can see why I'm not convinced otherwise.

If they pay me at contract rates double my current salary, and in my particular case my current IP assignment clause/the problem/ law makes it ok. Sure, pay me money to do something like what you guys need done.

Probably most useful if it's a contribution to open source.

No, the ideal interview is to ask them questions about what type of problems they've solved before and ask them to walk through what they did. Also, have a conversation with the person to get to know a little bit about what they're looking for in a job/company. It's an interview, not a tryout.

Metrics like a 10 minute task, or a 30 minute task is all relative. Do they know they language, the IDE, the operating system, the documentation, any experience in whatever abstract problem topic you choose, and personal comfort levels will all come into play.

If you want to filter someone to do a very specific thing then post to hire a contractor with the specifics of what you need. If you want a developer that can grow within your organization then pop quizzes area good way to dismiss good candidates.

> No, the ideal interview is to ask them questions about what type of problems they've solved before and ask them to walk through what they did.

This just doesn't work. Lots of people can bullshit very convincingly, and even if they can't produce novel solutions, can very well explain them in a matter that makes you think they for-sure know how to do it.

Hell, I can probably still prove lots of interesting things about set theory in math, as can many other people who studied math - doesn't make me the equivalent of the world-class mathematicians who actually came up with the proof.

Empirically, "talk to someone about what they did" doesn't give me hires that actually know what they're doing.

I don’t get why I have to prove to some strangers I can write code when I’ve been doing it for the last decade. It’s a ludicrous and broken system.

Because those strangers don't know you.

Any idiot can write things on a resume and say they did things they didn't do (aka lying). You would never do such a thing, of course, but as crazy as it sounds, there are people out there who would do just that! So because there's no professional license to write code, the only way to prove to these strangers that you actually can write code is some sort of exercise where you prove it to them.

I really don't get why this is so hard to understand, either. I get that live coding in front of someone else is a crazy stressful situation - I've failed multiple interviews because I couldn't perform on demand and answer the interview question in the interview setting, when I could easily have done so after taking a proverbial shower to have a think, so I'd love to get rid of them too. But unless we all band together and start a software developers guild or something, the live coding interview is here to stay. (Though, Triplebyte, now Karat, and others did take a run at improving the process, so there's that.)

I know what I know, but you don't know what I know. It's only by communicating, in a sufficiently unfakeable way, like a 45-minute in-person interview, that one can pass or fail the unwritten "can program" shibboleth.

My issue is that these coding interviews tend to turn up a lot of false-negatives and sometimes even false-positives. And of course they do. There’s so much more to what we do than being able to implement A* in 45 minutes without a real IDE could demonstrate.

You don’t get why you have to prove your ability to do a job when you want them to pay you to do that job?

Yep. I realize that’s not going to resonate with many or even most people, but that doesn’t mean I don’t believe it and that doesn’t mean it doesn’t have merit.

Law firms don’t ask lawyers to litigate in a mock courtroom setting before hiring them. Hospitals don’t ask surgeons to perform a little test surgery on a person before hiring them. Engineers aren’t asked to build a bridge to prove they know how.

I think the reason we are so unique is because we CAN demonstrate our ability to quickly and easily, and without risking human life. CAN of course does not equal SHOULD.

Law is a protected profession - you have to pass exams to be able to call yourself a Lawyer, and a law firm will verify that you have passed the relevant exams.

Same for medicine, engineering...

Any chump can call themselves a software engineer. That's the real problem here.