Totally, but leetcode wont filter them out.
Incompetent is fairly easy to navigate around, pathological bastardness is impossible to work past. I know that some companies use psychometric tests, but they are not based in anything remotely scientific, but are also super easy to game.
Coding challenges won’t filter every pathological bad hire, but coding challenges do catch a lot of bad hires.
LeetCode shouldn’t be the entirety of the interview. It’s just part of it.
leetcode doesn't catch bad hires, as someone whos done a lot of interviews, we could easily catch "bad" coders with a very simple whiteboard test or a "look at this code and tell me what you think"
but people being bad at code aren't the company killers, its the toxic people who are "good" at coding that do. They can be very good at these kind of tests, but are absolute shites to work with.
I think we actually agree more that I let on, and you are right that coding tests should only be a part of the interview process.
But in fact even in at-will situations, firing aversion seems to be a real and expensive issue that all kinds of managers in organizations face [1] [2].
So, in one sense I agree, bad hires are existential threats "only" because we don't fire them fast enough. On the other hand, have you ever had to fire someone yourself? Try it. It's heart-wrenching, even to a heartless utility monster like yours truly.
[1]: https://www.econlib.org/archives/2012/06/firing_aversion_2.h...
Well worth it. Highly recommend these tools to play the dumb interviewing game.
However, I have a real question as a follow-up: Have you been able to profit from this skill? If true, you should be able to crush any "whiteboard" programming interview. If I had it, I would move every 2-4 years and get large pay rises. You should definitely be able to pass FAANG interviews with that skill.
> Many of those questions were thesis/publication-worthy decades ago when first solved
I do work in a team where majority was hired by puzzles of sorts. All of them are smart. They are not software engineers and it shows massively.
Coming from Real Engineering and making 2x more programming, I lol at people who say there is any Engineering in software.
We are programmers dealing with layers upon layers of abstraction. Knowing to optimize for time by using vectors is more of an art, than a science.
I did do safety critical C code and Assembly which are probably my hole in the whole 'its not engineering'. But javascript/python/backend work: programming.
It's just a job title, arguably one without the protections or standards afforded to chartered engineers, and with an incredibly low barrier to entry. Pretty much the reason we have to do this ridiculous dance with tech tests, LeetCode, and 12 stage interview processes.
Exactly my point.
The strong consensus was that software is an engineering discipline, and the nitpicking you can have on any particular topic that is/isn't engineering-y can be applied across the board.
But this conversation is totally moot IMO because if you have a bachelors in computer science or software engineering, and a masters in the same discipline, and several years experience, you can apply and become the same chartered engineer from the same association that does "real" engineering.
I see it as a difference in time and stakes.
We’ve built bridges and boats for thousands of years, where the outcome of failure is people die.
This has a lot to do with why it’s easy to estimate the time and cost to build a house, but it’s a rare shaman who can consistently estimate software projects well.
Once we’ve built software for even a few hundred years, we’ll probably have it pretty dialed in, too.
We write software that has killed people due to bugs too, such as Therac-25 releasing too much beta radiation killing patients.
> This has a lot to do with why it’s easy to estimate the time and cost to build a house
My parents and in-laws are property developers and I've also been involved in a build; not one was on time or budget. Some are really off budget, beyond the scale software projects that mess up are. In China they've had projects run into the billions and be decades late.
I do understand your points, but this is what I mean when I say you can nitpick any issue with software not being Real Engineering and apply it to other engineering categories too.
But yeah we may have a very different process in 50 or 100 years. I'm only 10 years into my career and it's already pretty different to when I started!
One related problem I see is that Computer Science tends to be under-estimated by other engineering fields. A ton of software engineers did not study software. They learned to program as part of their engineering studies, and they believe that they learned both Computer Science and their engineering topic at the same time.
I may be biased, but I see fewer computer scientists pretend to be mechanical engineers because they once tuned a PID.
If Bachelor of Science computer science grads were that much more extraordinary, they wouldnt be working side by side with non-degreed programmers.
Chem Engineer here, I wrote random forest once. My coworker with a CS degree, all front end work.
There definitely is. Safety/mission critical code is definately a more rigorous, formal process where you don't just download a library for padding a string (you probably wouldn't use built-in strings anyway on this kind of code).
But I agree that the way most coding works (i.e. move fast and fail often) is completely antithetical to how engineers should work. But there's infinite moneys and almost zero accountability, so who's gonna stop them?
>Knowing to optimize for time by using vectors is more of an art, than a science.
that's definitely why I fully agree with calling programmers creatives over engineers. once you get past dead simple problems, no two programmers are going to approach a prompt the exact same way. performance, scalabiliy, documenting, version control friendliness, even ephemeral stuff like code style. coders very much are artists who happen to know a lot of math.
It is a downgrade in my opinion.
Most programming work needs common sense a lot more than it needs theoretical algorithmic knowledge.
And almost nobody values good documentation, especially as a competetive advantage.
FWIW I think LeetCode problems can make sense for some businesses, but most not.
But it's entirely pointless.
And because they ask pretty irrelevant questions, the 6 weeker will do phenomenally, staggeringly, profoundly better then the recent Stanford PhD who has been studying a specific problem for 5 years or the recent minted multimillionaire who spent the past 10 years building and selling successful software companies.
Those people will fail and your 6 weeker who's never heard of cmake, diff or gdb will pass.
There's once again, an entire industry designed around how insufficient and hackable this stupid hiring practice is.
When I see there's leetcode as the hiring process I presume the place is full of bozos because the process actually strongly preferences them.
Either it wasn't a true "zero" but rather they were really good e.g. at advanced math or physics and managed to see some parallels, or it has to be a genius. I'd be very interested in hiring such a person.
The code to solve basically all of them look nearly identical after you classify it and the questions asked (what's the big O of this) are also all identical.
It's an extremely narrow and teachable class of problems.
There's outliers for sure but the 80th percentile are basically just the same damn things slightly rephrased. It's extremely gameable and doesn't test the kind of breadth say, fixing a crashing program would show.
Here's a list of 16 https://github.com/Chanda-Abdul/Several-Coding-Patterns-for-... ... That's the 99th percentile.
If you think someone passing that means they can fix the conda bug in your k8s cluster or whatever real problem you have, dream on. It'd be like hiring a cook by quizzing them on the names of kitchen utensils.
The other way round (people who code, but can't leetcode) I agree it's an issue. Unfortunately as noted here in comments, cost of a bad hire is much much worse than cost of a no-hire, so it's the price I'm willing to pay.
Excellent! That’s a something, a something you can measure. Back when I was involved in hiring, it was definitely better than not having that (at junior levels).
However, the idea that “IQ is all you need” is so obviously false that I think you need to reconsider. Bill Gates famously, (infamously?), hired for IQ. Microsoft used puzzles as their IQ proxy, and I think leetcode is just a modern variant. Possibly a worse version, as training has a bigger effect.
Bill Gates later would claim that he overvalued intelligence. I would say that intelligence is necessary but insufficient. We would get further, faster, with a quick IQ test and then dig into the candidates experience, communication ability and personality. No need to study!
The FAANG companies all have a line a mile long of candidates willing to put up with any amount of BS for a chance to work for the name and $$. If you aren’t such a company, trying to hire that game changing person for $$’s you can afford to pay is going to require a different approach.
I think the mass trained at bootcamp is an exaggeration. The best bootcampers are those that use it as an extended education from learning the traditional ways. I'm sure most people here complaining about leetcode can indeed spend 6 weeks there and be interview ready. But few have that time nor spare cash lying around for such a venture. And why should we need to spend thousands just to ace a quiz like it's some SAT?
>cost of a bad hire is much much worse than cost of a no-hire, so it's the price I'm willing to pay.
At this point I wonder. Hearing way more stories about "survivors" from layoffs burning out from jobs that keep promising that they'll hire more staff to help out with. When in reality they have a hiring freeze or are trying to offshore the work that ends up needing more time to be fixed.
The stalls in a no-hire aren't just costing money in the lack of productivity, it's costing currently working employees who are doing 2,4, 8+ jobs without any meaningful increase in role/salary. It's not sustainable. You're draining a little bit of blood from a rock, but the rock is clearly starting to crumble.
If I genuinely wanted a job again all I'd do is go to one of those drilling game websites for a few nights and then say whatever the interviewer is looking for.
Any test I can fail one day and pass 3 days later that is ostensibly supposed to assess whether I have skills that take 15 years to master is total bullshit.
The initial stated problem was different: false-positives, bad engineers practicing for 6 weeks will pass the test. That I seriously doubt is a real thing. If they managed to learn that fast, they must be good. They could be junior-good -- that's perfectly fine, you need to hire juniors too. You don't judge seniority solely based on leetcode results.
I could be wrong, but I think dynamic-programming problems belong in that list.
Recently I failed the first round of a fintech analyst position because I didn't complete the 9 undergrad (or high-school, in certain talented schools) math problems (I have a math PhD) in the 1 hour time frame. In retrospect, I should've cheated using computer/online assistance, and my true failure was my unwillingness to cheat. Not because fintech analysts are cheaters, but because the testing process is nonsense, so recognizing that, one should cheat it (indeed, an analyst mindset!) I'd rather be jobless than to adopt that mentality though. (and in fact I am, the only job I was able to find was a minimum wage service-type job, it can't even pay for daycare, I'd rather raise my child on my own while wife works thank God.)
We have high-interest rates and global crises resulting in companies not hiring while simultaneously governments give orders to news media to tout the strong economy. Ineffective hiring practices will always exist, but people don't complain when jobs are available; you only see this type of discussion online when things have gone sideways.
Integrity is really tested when it costs you something. I'd call this a true win.
(I recognize there are dire situations where cheating might be warranted. I hope you're not there yet.)
I've seen this reasoning, but I'm wondering if this is actually true, especially if you also believe the other myth in hiring: most people applying are not qualified for the job. Since most hiring actually requires you to fill the position in the end (in other words, you can't leave it open indefinitely if no one qualifies as eventually the pressure to hire mount up), every false negatives does increase the odd of you hiring a false positives as well. Just take the example at the extreme, if there is only ever one person on Earth qualified for your job and you got a false negatives on him, there is no other way but getting an idiot in the job.
Someone who does well on leetcode is usually easy to teach so they will become good/great in a year or two, but if you don't have that time then go for leetcode+experience just like everyone else is.
We would work together at the whiteboard, with me teaching them the basics of our application domain and our internal distributed computation framework. Then there were a series of 5 increasingly difficult iterations we'd work through. First, do a simple calculation. Now there's a need for a more difficult computation. And an even harder one. Then explain that it needs to run over billions of log lines, and it's I/O bound, so let's make it faster. Finally, a problem that requires them to probe me deeply about the nature of the data and only needs a hand-wavy answer (calculate a median in a single pass with limited memory - not too bad once you extract from me that the samples are all integers between 0 and 500). To the candidate, this looked like iteration driven by evolving requirements, not 5 interview questions.
I mostly let them drive, but I jump in now and then to keep things on pace. And I would try to answer any question the same way I would as a teammate / mentor.
They don't need to get every problem right. The increasing difficulty is designed so that almost everyone gets something right and something wrong (< 1% gets all the way through the last one). That's an opportunity for me to give supportive feedback to see how they respond. The best responses to feedback yielded something like "oh cool, I can also use that over here to simplify the code". The worst was someone insisting that you can flip the order of addition and division without changing the answer (i.e., order of operations doesn't matter), me saying "I think it does matter, try doing 1/2 + 1/4 in different orders", and them saying "no I have a math degree, I know what I'm talking about."
What they do need to do is be able to solve problems collaboratively, extract learning from an eager mentor, and extrapolate from those learnings to solve problems. The best candidates start finding trade-offs across the application domain / tech tooling boundary that I haven't even brought up.
One of the downsides is that it takes a while to calibrate as an interviewer, and that calibration is not entirely transferrable across interviewers, as candidates will respond differently to different interviewers. I did this interview thousands of times (my record was 14 times in one day, and frequently 20+ times a week - we were scaling an org from ~50 engineers to over 1,000). I never stopped learning about signals I was seeing, but I was mostly calibrated after a few dozen candidates. Leet code calibration is much more transferrable, but what use is that if it's not measuring what matters?
One thing that I think leet code interviews get very wrong is that interviews involve two sides trying to evaluate each other. Wearing people down with leet code is not a good pitch for your work environment. Many candidates told me at the end of my interview that they were shocked at how much fun/educational an interview could be, and many hires said that this interview style was the reason they accepted our offer over more lucrative offers, because they felt they would learn more and have more fun with us.
> One thing that I think leet code interviews get very wrong is that interviews involve two sides trying to evaluate each other. Wearing people down with leet code is not a good pitch for your work environment.
Depends where you’re looking in the hiring pipeline. If you’re looking at raw applications submitted to a publicly listed job posting, I can say without reservation that the majority of candidates are not qualified. The pre-screening candidate pool is extremely bad on average, especially for remote job listings. Some people spend all day clicking the apply button on every job listing they see.
Your post constrains another misunderstanding about the hiring process: The goal isn’t to hire the first person through the system who is good enough to pass some arbitrary threshold. The goal is collect a number of candidates and hire the best one. People hate this reality, but it’s true.
I definitely do not have any misunderstanding here, and my description of hiring is the same as what you wrote, just in different phrasing: after a period of time, the company has to pick a candidate, presumably the best one by their interview signals.
I am disputing the standard claim of "it's better to have false negatives than false positives" by pointing out how that mindset does not work: since you have to choose someone anyway, the whole "avoiding false positives" might not be a thing at all
Or rather: the best fit. You don't need a Turing Award winner to clean up your website.
As someone who’s done a lot of interviews, this is very true for SWE roles.
I've never been through the "regular" IT hiring process myself, but I've interviewed several candidates for an Android developer position. I liked asking questions and looking at how the person thinks. I didn't expect exact correct answers for every question, I just wanted to see whether they have a fundamental understanding of Android. After that, they had to build a small app at home (not my initiative) to show their skills.
There are better questions where I do think google should be allowed but they aren't LC-style. Some examples are building APIs or operating on files
“Trick” questions are dumb, but you want to get an idea of if the candidate knows their stuff (and let the candidate know what we’re like — interviewing is a sales process in both directions). If someone asks a question like “is it ok to modify the argument” (or says “I’ll assume I can’t modify the argument”) that’s great (or says “I can’t remember if strlen includes the 0 byte so I’ll add 1 — normally I’d look it up”) - again, no trick questions.
When I say “simple” it’s something like atoi or “shuffle a deck of cards”: basic, but not 100% trivial like fizzbuzz.
On of our best hires was a guy who made a fundamental mistake — when we asked him if it world work as intended he said “I’m a fucking idiot” and fixed it. You’re not supposed to talk like that in an interview I supposed but we read it as a strong positive signal.
We had one candidate who insisted that parsing a string and returning an integer was an unreasonable question because there was already a library function for it. “But what if you have an embedded system and no library?” (an actual situation for us, for part of our system). He was adamant — honestly that was a failure of the phone screen: had it been caught it would have saved us, and him, from wasting time.
It seems that “able to code a simple problem well” is a far less ridiculous proxy for good software engineer than “able to code a hard problem at all”. Much to my surprise.
The software industry is notorious for underestimating overall complexity factors, understanding uncertainty, and time to manage them and interviews seem to be no different. So “simple” problems are often good (I agree with you) because they’re actually not so simple for most people when you factor everything in, they’re often reasonably challenging under the set of circumstances.
Meanwhile genuinely difficult problems (in any arbitrary setting and longer timelines) require a whole lot of rote training and reusing little clever idiosyncratic techniques that may or may not be generally useful, so you very often end up with a question lottery on whether you already know how to address the problem in question or not correctly vs having some problem where a reasonable solution and expectations around that can be derived in the time (and environment) it’s taking place. Lots of nonsense.
Truth is, I'm never looking for "the best", I'm looking for "good enough to do the job competently". Because we can hem and haw over what is best for hours. And we could disagree with what qualities in what proportion make someone "the best". But we can all generally agree with "could do the job". As I've mentioned elsewhere, we're hiring right now and we have the duality of candidates in our pool.
There's one guy who is just a hard no from at least two people in the group. And that translates into a no from the third member simply because we're so adamant that he's not the guy. And there's another who we're all feeling really solid on. Not "the best", but solid floor and we feel they'll be able to learn.
But being able to weed out people who can't even perform the basics despite having a decent resume is invaluable.
When I worked for more medium sized companies, I'd get software engineering candidates who obviously did not even remotely know how to code. Imagine the worst possible software candidate, and then lower your expectation even further. One couldn't even tell me what an "int" was. I think a single "for loop on a whiteboard" question would filter out at least 50% of candidates.
"You can lead a horse to water but you can't make it drink."
It says barely anything about the universities and everything about the student. If a student just wants to just have a perfunctory gargle at the fountain of knowledge instead of drinking as deeply as they can, that's on them.
Almost everyone embellishes their resume. Some do it to an extreme. The first stage should be weeding these people out, and any hiring manager learns this lesson very quickly.
Story: I learned this as a hiring manager early on. I was the resume screener, first contact and then last contact. Everyone in between was my team and other adjacent teams. I failed to ask the right questions in a phone screen and when we brought this guy on-site he failed very bad. Even worse, he broke down begging for for another problem he could solve.
It was bad for everyone and we were a very accommodating group that gives many tries. We weren't looking for perfect answers to one question.
Though I still do start with a short fizzbuzz-level question because you would be surprised how many people fail that and it makes it way less awkward if you start with atoi and they can't write a loop.
If they fail fizzbuzz I generally just turn the interview into a chat or give them easy questions to practice, rather than say "yeah no thanks", because I feel like the latter is a bit mean, and everyone can benefit from interview practice.
It’s kind to politely ramp the interview down in that case but I’ve learned to then cut off any later interviews. It’s a waste of the company’s time and the candidate’s too.
If we were going to have lunch the recruiter takes them to lunch anyway, unless the (now former) candidate doesn’t want to.
> We also used atoi/itoa
Trying to debate the interview format or reject interview questions is one of my hard-stop rejection triggers in interviews now.
The couple times I’ve been part of hiring pipelines where candidates argued about the interview itself and then got hired anyway, they became extremely difficult employees within the company. Arguing about interview questions turned into arguing about every other ticket, code reviews, and every architecture choice.
I’ve had the same experience where we take an actual problem we solved in our codebase and turned it into a small interview problem. Same thing: Some candidates will try to debate the relevance or try to wriggle out of it.
if the interviewee pushes back and find they don't like the answer, then it's a good signal to both parties.
Sometimes you just have to "document your objection, but do it anyway."
Sometimes we want harmony and not drama.
Knowing how to push forward without being abrasive is a more valuable trait.
Fwiw this is surprisingly subtle and non-trivial to get right. A naive "random swap" approach does not produce uniform permutations. Sorting based on a random comparison function can likewise produce biased results depending on the implementation of the sort. Tagging each element with a random number and then sorting based on the tag works (or so I think?) but you need to deal with the case where tags collide. Fisher-yates shuffle is bulletproof but I would not trust myself to write it from memory lest you end up writing Sattolo's algorithm instead.
At the end of the day the question is: “could this person do the job and could they work with others in the process?”
this would make sense if they filled these positions in 2,3 weeks tops. But that's the insane part; I hear of interviews going on for 6-7 stages now, lasting almost 6-10 weeks. At that point I feel you are no longer avoiding false positives, but simply hiring the most desperate programmers willing to jump the hoops. The best candidates will get an offer mid interview at that rate.
>But just don't! It's fine! just move on,
I've been "just moving on" for some 9 months now. There's only so much morale in a tank, no matter how BS you know the whole process is. I just wanna move on in life, not be bogged down on how many permutations you can color an N x M grid with.
> I hear of interviews going on for 6-7 stages now, lasting almost 6-10 weeks.
Best tests are simple and common computer science problems in algorithms and data structures with no tractable solutions. There is no facile answer, you truly have to reason from first principles for any particular scenario because there is no alternative.
The side effect of this though, is that it's an extremely painful process for the applicant. Even if you're a great software engineer, you're going to freeze up, miss something, have a bad day, and fail a few interviews. Now for most software engineers failure is an unusual and harrowing experience, so they react really badly to it. Especially ina scenario where it's difficult to blame anyone but yourself. But just don't! It's fine! just move on, there's plenty of jobs and the interview process is a blunt tool, not a final evaluation of your worth in life.