Yup that's what it's about!

The problem with trying to test how good you are at the job is that you simply can't do it in a 1 hour (or even a 1 day) interview. You can however assess how smart somebody is, which is definitely correlated to job performance. It's also connected to growth potential - even if it were somehow possible to accurately measure how good a programmer a candidate is at the moment they are applying for the job, companies would still want to try to predict how good they are likely to be over the next few years.

Are you saying the Leetcode-style interview assesses how smart someone is?

If you don't get the answer, you might still be above that smartness threshold, but didn't get it for some other reason (didn't have time to study, didn't sleep enough the night before, interview anxiety, etc.).

I know we try to stay away from the “we’re smarter than you” vibe here because it’s gross, but pretending leetcode style questions don’t require some degree of complex thought is absurd.

I think it's unreasonable to expect people to spend their off time practicing these sorts of problems, in the same way we think it's unreasonable to require everyone interviewing to be working on personal development projects over the weekend. I suspect most working people simply do not have the time.

Nothing tells me more about someone that is full of it than simply being dismissive of others. I ask for an example to support your argument, instead you chose to be defensive and give me attitude.

Why not just give candidates a standard IQ test then? They are probably more reliable than Leetcode...

leetcode is the SAT for coding, not perfect, but at least it's close to fair play.

this is what everyone said a college degree was for. Or experience. Neither of which matter when a senior dev with 10 years of experience and two kids still has to find an hour or two a day for three months in a row to grind out textbook algorithms to fake problems. Just to switch their fucking job. Thanks to cargo-cult insanity, god help those stuck in miserable jobs that just want out.

Yes, they're rarely encountered in most day-to-day jobs. But that doesn't make them fake...

The actual solution for encountering a differential equation at work is wolfram alpha, not getting a pen and paper to apply heuristic knowledge on solving differential equations

It would be amazing if leetcode were like the SAT, and you could just get one good score and then never think about it again.

Anything like that would make it much lower-friction to switch between FAANGs (and friends), though, which I suspect is a big part of why they've settled on doing things this way.

In fact I believe software should have some qualification tests, e.g. general coding, database, cloud computing, etc. Like CPA for accountants. Each test should be valid for a few years in each category.

What some of the people in these comments seem to want though is some sort of standardized score/ranking system, which I suspect might lead to even more nightmarish outcomes than the current leetcode-y interview processes. (e.g. Employers setting absurdly high cutoff scores, choosing one applicant over another because simply because they scored a couple points higher, applicants grinding unimaginable amounts of unpaid hours to bump up their scores a bit, etc.)

As many other comments have indicated right here on Hacker News, professional licenses are commonplace for occupations such as plumbers, electricians, doctors, lawyers, and even many types of engineers.

I suppose that central governments (such as the US federal government) should offer licenses for myriad types of software engineers, hardware engineers, software architects, hardware engineers, and so on.

Wouldn't it help companies if they could choose to interview only candidates who were licensed as, say, a level three penetration tester (intermediate penetration tester) or a level five database architect (expert database architect)?

The whole "Let's reinvent the wheel mentality" surrounding software, The Internets, and hardware simultaneously bemuses and frightens me.

"The eye never has enough of seeing, nor the ear its fill of hearing. What has been will be again, what has been done will be done again; there is nothing new under the sun." King Solomon, Ecclesiastes.

and

"I only wish that wisdom were the kind of thing that flowed ... from the vessel that was full to the one that was empty." Plato, Symposium

Why don't we simply allow unqualified, blind, inebriated people to drive automobiles on public roads at whatever speed they would like? Why don't we let a guy who watched a bunch of YouTube videos call himself a brain surgeon, and perform brain surgery on people who don't even need brain surgery in the first place? Hey, wait, i've gotta gureaat idear: y botherr haviing aany ruuules at al! Sheesh.

Without rules, men simply return to a state of nature where life is short, brutish, and mean (Hobbes).

I just found this on Google...

****** Origin of Life is Nasty, Brutish, and Short

This expression comes from the author Thomas Hobbes, in his work Leviathan, from the year 1651. He believed that without a central government, there would be no culture, no society, and it would seem like all men were at war with one another. ******

The “Wild West” mentality of folks who seem to believe that rugged individualists (not federal agencies such as the US Department of Defense) built Silicon Valley, is perched atop the same type of popular, yet nonsensical, mythology (falsehood) as Horatio Alger's famous character who was actually named Ragged Dick. (Really, Ragged Dick was the character's name, I am not being facetious) who metaphorically pulled himself up by his bootstraps to rise from street urchin to CEO (who was a wealthy industrialist).

Imagine a military, any military, anywhere, anytime in human history, that didn't have ranks, titles, and gasp... tests which members had to pass to move up the ranks. How well do you suppose a military without ranks and without tests would fare in combat? Obviously, such as military would be in a state of hopeless disarray.

Licenses are not a necessary evil; they are a good, and proper way to identify and reward qualified professionals, while simultaneously enabling "the rest of us" to know, for example, who's a mere private, whom we can walk past without batting an eye, and who's a colonel, whom must stop and salute.

Imagine a hiring manager say, "Hey, this kid never went to college, but he's a freshly minted level one software engineer, I say we bring him in for an interview."

Why should every company need to create their own initial screening tests? Imagine a trucking company that needs to hire a truck driver with a particular type of commercial driver's license (CDL). In the employment advertisements they post, such companies almost invariably include verbiage such as, "Class A CDL required" or "Must have a Class B CDL." See? The candidate must have already passed an initial screening test, by acquiring a particular type of license, prior to being granted an interview.

This is obviously a huge benefit to both candidates and companies alike because it saves both sides a lot, and I mean, a lot, a lot, a lot... of time!

These days it is comically inane that tech candidates are normally expected to take an endless stream of initial screening tests to prove their mettle to each and every prospective employer (unless they were referred, famous, or for some other reason exempted from the requirement).

Imagine a CPA (certified public accountant) being required to pass a basic auditing test before he was granted an interview for a new job. Why would a company ask a CPA to take such a test? If a candidate is a CPA, then (unless, for example, he cheated on his CPA test, or suffered some sort of memory loss) he has already proven that he has a substantial amount of knowledge about auditing.

Yes, of course companies should administer their own tests. But professionals licenses can, and do, enable both candidates and companies to avoid the sort of initial screening test which companies commonly require of software engineering candidates. Currently initial screening tests, such as LeetCode, are a response by hiring companies that are typically deluged with a sea of unlicensed (and almost entirely unqualified) candidates, all of whom claim to be qualified.

There are React Boot Camp folks who can whip up a frontend in no time who are arguably more competent at this task than a PhD in Comp Sci, and there are PhDs in Comp Sci who never wrote a lick of code in their life.

Technology in software IS Political, and 30 years of development experience shows me various parallel worlds that are arguably better than our current one.

Someone who grew up coding and loves hacking as close to the bare metal as possible is worth more than a dozen of your certified software engineers.

Let me know when the world standardizes on a Comp “Sci” curriculum.

Actual computational science has very little to do with computers and software development, which is a thesis statement I will be happy to support if pressed. For now, I end my rant

How is it fair, it favors young graduates and people with lots of time to prepare tremendously. Would it seem sensible to you that every time a doctor applied for a job, even if he has 20 years of experience, he'd need to compete with recent medical school graduates on some first year medical school exam?

But if we're saying Leetcode is simply testing for IQ and not really for software engineering ability, why not just test for IQ? IQ tests are designed in a way that after a certain threshold it's pretty hard to improve in them - they actually test some innate ability. Leetcode tests how well you are prepared for Leetcode, and perhaps how well you do under stress. It correlates only slightly with intelligence and even less slightly with programming ability.

In other words, whatever it's intended to do, one of its primary functions in practice is to screen out people who are bright, driven...and poor, working long hours and trying to keep themselves and/or their families going.

That it's a bit of a sacrifice is the point. If you're naturally smart enough this stuff comes quickly, great. If not, you're going to have to work for it - that requires discipline most people don't have.

Better put, smart and/or determined to succeed.

Whether you realize it or not, you're advocating for keeping people who aren't already mid-to-upper-middle-class (among others) out of these kinds of tech companies. Anything that's designed to make you work hard extra, outside of work to learn separate skills just to pass interviews is guaranteed to make it disproportionately harder for people who are, for whatever reason, unable in practice to devote many hours of their free time to fairly complex technical studying.

No matter how "disciplined" they are, people already working 80 hours/week just to put food on the table don't have the luxury to be doing that. No matter how "smart and/or determined to succeed" they are, people raising 2 kids by themselves would be irresponsible to be doing that.

Now, maybe you think that kind of person should be denied an opportunity to join the super-1337 hackaz' club that is FAANG or whatever. Personally, I think that kind of classist gatekeeping is disgusting.

Ah, you're talking about those programming jobs that never require thinking about algorithms?

I've been working as a programmer—largely PHP, Java, Objective-C, and Swift—for over 20 years now, and never, in my work, have I been asked to invert a binary tree, reverse a singly-linked list, or any similarly contrived scenario.

There's a huge difference between "thinking about algorithms" in the most general sense of that phrase (which means basically any programmer who does any of their own design work) and in the sense of the particular "algorithms" that we learned in CS classes and are featured in Leetcode problems.

Search engine: read material about the heart of the problem from domain experts, try different approaches and then decide.

Leet Code exercises are a blight on our industry.

If you want a shibboleth just say so.

I once interviewed a guy- a CTO at a biotech- and he wouldn't answer the question "I have a million DNA sequences and want to count the number of occurrences of each sequence" (could be hash table, could be any number of other solutions; he just balked and exited the interview).

Only if the person can get all that right would I even consider asking something more complicated.

Maybe he's just been around the block a sufficient number of times to have gotten tired of the standard SV "here's a hoop, jump through it, now do it again, and again and again" ritual that it likes to call "interviewing" for some reason.

Seriously - your questions are fine for junior or mid-level roles, but beyond that, you should have more important thinks to talk about. And both of you should be able to tell if the other is a bullshitter and/or a lightweight within about 30 minutes at most (and often far quicker than that) of a normal, focused technical conversation -- without having to specifically grill the other person, or otherwise put on the pretension that you're one whose bona fides are established beyond question, and they're the one who needs to jump through a sequence of hoops to prove that they are worthy of your time and attention.

Just cut the crap, get down to brass tacks, and talk what needs to be done as if they're a peer. That's all you have to do.

So usually I end up after 45 minutes finding that the candidate has more or less tapped themselves out at "make a hash table of string keys, use it to store counts" (OK for a very junior programmer) or "make a perfect minimal hash" (I help them get there if they don't know what those are) or "use a probabilistic counting filter". This is about all I need to make a determination.

If all you do in my interview is know that a hash table or other associative data structure is good for maintaining counts, that using a full ASCII byte to store symbols from { A G T C } is wasteful and compress the letters to 2 bits each, you can pack multiple 2 bits into bytes, and can write a function that codes and decodes such data, I'm happy and you get a pass. I'm even happy to sit there and help people through nearly all the steps of the codec. Not trying to trick anybody or select for obscure CS knowledxge.

To me the real question is, how much hinting is reasonable for the coding and decoding function? Many programmers (including senior ones) struggle to implement:

x <<= 2 # left shift the int to make room for the next item x |= pattern # or the new 2-bit item into the accumulator.

Since I never use bit munging in my day job (it's 90% python data science) should I really ding somebody for not knowing about left shift or or-assignment?

What I really don't get is why people immediately jump to trying to implement huffman coding, RLE, or lookbacks as a solution to reduce the size of the DNA string. I still wonder if how I present the question is giving everybody a fair chance to shine.

I think "key/value counts" is absolutely the correct starting answer. Anything after that is optimization. In the real world optimization comes at length through research, learning, grokking, reading, benchmarking, sleeping, mulling things over. The only way to short change that is knowing about those optimizations in advance, so might as well just ask about those algs specifically.

All that being said, I think it's an enlightening question. Thank you for sharing it! You've educated me. I've been coding a loooooooong time and didn't know about bloom counting filters specifically.

https://bmcbioinformatics.biomedcentral.com/articles/10.1186...

With Leetcode, you have to retake the test multiple times every few years when you change jobs for each company you apply to.

Leetcode would be a lot more tolerable if it was administered more like the bar exam or like medical exams. Whatever happened to DRY?

Which means... Always. Be. Leetcoding. Do the bare minimum at your job and then do LC the rest of the time. Because your job is just your job, but your career is LC. These companies don't yet realize they are optimizing for mercenaries that have no loyalty to the code nor the company.

On a slight tangent, some of the people over on Blind would sell their own mother for a tiny bump in TC. That mentality used to be limited to Wall St. or maybe Big 4 accounting firms, etc. But now it's the whole tech industry. I remember having a software job was a lot more fun, back around 2007ish. That culture is so foreign to me now.

Even having to pass such a test on a very aggressive set schedule—say, every 5 years—but only at those set times, would be better—for developers. Not for the companies hiring them.

Being able to quickly invert a binary tree and that kind of nonsense belongs in library code (at best). It's not something that the average developer should ever need to worry about the actual implementation of, let alone have to implement from scratch in under 30 minutes.

My coding question, for example,

You have a music player that should be playing a 900 song list randomly. You notice that you keep hearing a song being repeated during your drive and you are curious if its truly random. You also keep hitting skip in the hopes a particular song comes up. Write a piece of code that simulates this.

You should tell me three things; 1) the number of songs played before a repeat occurs 2) how many songs needed to be played before you played them all 3) after you succeed in playing them all, how many times has the most common song been played.

You can use libraries if you know them, and you can use the built in sort method in the language, I will google it for you if you don't remember the syntax.

This is the insanity to me, "Here, do this contrived task that doesn't represent anything you will be doing... to prove that you can do the job"

I once had a whiteboard interview for a senior engineer position where they demanded that I write it in syntactically correct python, indentions and all, on the whiteboard. I'm trying to talk about code at a high level with them meanwhile they are deducting points because I assigned a dictionary key directly vs. using the dictonary's method. It turned me off to the company as a whole and the entire interview went downhill from there.

Yes? Yes. Figuring out how to make a computer solve problems is very much the job of a software developer. They will only encounter harder and less well defined tasks in their actual job. If they can’t do this and you hire them that is like hiring an opera singer who is mute, or a baker who is deadly alergic to flour.

> putting them on the spot in an already tense situation and expecting them to code while you watch

There are mitigating factors one can do. We make sure our hiring managers let the candidates know that there will be a coding challenge. We ask the candidates if they prefer to chat while they work through the task or prefer to be left alone and we acomodate what they choose. We let them know that whatever style they prefer it won’t change anything.

> meanwhile they are deducting points because I assigned a dictionary key directly vs. using the dictonary's method

That sounds very unpleasant. Sorry to hear that. Interviews are a two way street. You are interviewed and at the same time you are interviewing them. I think you were right in judging them, and you dodged a bulet there.

By the sound of it you are a talented, and capable developer. It might be that you can’t imagine it, but there are people who apply for developer jobs, has a really good ability to talk about the job, they seemingly have the right experience, yet somehow they can’t program even super simple tasks. Even after you give them every acommodation immaginable to humankind. If you haven’t seen this yet you won’t believe it. If you have seen it you want a filter against this particular kind of candidate.

I’m not saying that this filter goes always well. Every filter ever invented had both false positives and false negatives. We might lose a briliant developer because some quirk of the task throws them. It is sad. We are trying to minimise the chances of this, but it certainly happens.

Yes.

> I'm disputing is your assertion that there is a correlation between doing the contrived problems under unrealistic conditions and future job performance.

You are projecting something here. You are saying, without any supporting evidence, that the problems are contrived. They are not. They are really the core of what me and my coworkers do day in and day out.

You are also saying that the conditions are unrealistic. What makes you think that?

You are right though I have no evidence they are contrived other than your example problem. The unrealistic working conditions though is indisputable, do you typically relay your google search's for syntax through your boss? Do you often work under extreme time pressure on problems you have never seen before? Perhaps you do, but I can tell you with certainty that it's not the norm in the industry to work in that way.

Indeed, this is a holy grail of the applicant funnel. If a tech-for-interviewing company comes up with a better way to remove the worst 50-ish% of applicants efficiently, they’ll have no worries about their future. (The overall applicant pool is significantly adversely skilled as compared to the have been or will be quickly hired pool.)

And they are, frankly, pretty bad at writing code because they have little empathy for the reader and greatly inflated sense of self worth. They’re the kind to use complicated C++ features or algorithms for little reason. Essentially, smart idiots.

They also believe in silly things like LC being a fair and rational way to evaluate candidates and don’t see the bias at all. “Eugh she’s an ugly woman, I think I’ll give her the LC hard and little help.”

There is another class who realizes how stupid LC is, but are happy to play the game to quickly accumulate power and prestige. They usually have psychopathic tendencies and aren’t great coworkers.

LC is great at hiring these types. Feel free to stick to it if you enjoy having them as coworkers.

I'm outside any major metro area, when we've needed to hire there haven only been a handful of people responding. When people come in for an interview it's been pretty easy to tell if they really can't code at all without making them actually code. I haven't found this to be challenging.

I suspect the issue is interviewing a large number of people in a short period of time. If you don't have 45 minutes or so to spend on each person, using automated leetcode style problems probably starts to seem like a pretty attractive way to weed people out.

Lastly, usually one or two people from my team would take part in interviews. If there aren't any developers available at interview time (i.e., it's a manager and a someone from HR), I can start to see how people without any real coding experience can make it through the process. Again, this is probably a place where automated testing looks like a reasonable solution.

Eventually, yes. But it takes time for them to start, time and money to onboard them, evaluate how they’re ramping up, then if not acceptable, to follow whatever performance management process is indicated by the company and local law, then transition whatever work they were doing. This could be several months and tens of thousands of dollars just to get back to a worse state than when you walked into interview that candidate.

Interviewing even slightly better than last year can pay large dividends.

  # You have a music player that should be playing a 900 song list randomly.
  # You notice that you keep hearing a song being repeated during your drive and
  # you are curious if its truly random.
  
  from collections import Counter
  import random
  
  r             = random.Random() 
  songs         = range(1,900)
  unplayed      = set(songs)
  
  # You also keep hitting skip in the hopes a particular song comes up. 
  # Write a piece of code that simulates this.  You should tell me three things;
  # 1) the number of songs played before a repeat occurs
  # 2) how many songs needed to be played before you played them all
  # 3) after you succeed in playing them all, how many times has the most common song been played.
  
  counter         = Counter()
  firstrepeat     = None
  totalplays      = 0
  
  while unplayed:
      song = r.choice(songs)
      unplayed -= set([song])
      if firstrepeat is None and song in counter:
          firstrepeat = len(counter)
      counter[song] += 1
      totalplays += 1
  
  print(f"""
  Number of songs played before a repeat:             {firstrepeat}
  Total number songs played:                          {totalplays}
  Most common song:                                   {counter.most_common()[0][0]}
  How many times has the most common song was played: {counter.most_common()[0][1]}
  """)

  $ python3 foo.py

  Number of songs played before a repeat:             50
  Total number songs played:                          7263
  Most common song:                                   375
  How many times has the most common song was played: 17

One other thing, I just got a new car, and this was the first time I was using 'next' using bluetooth. So, as I was driving in, I was in my head trying to figure out whether the phone now only had 100 songs downloaded, or whether this bluetooth 'next' function on shuffle was picking a random song from the list, rather than 'next' on a shuffled list.

You might as well just stand behind them and breath down their neck for added effect.

Seriously, that's way too claustrophobic, and not to mention a huge practical annoyance -- for the added latency of having to ask you to google stuff for them and then give you the result somehow, instead of just letting them do it themselves, when all they want to do is get past this trite exercise and start having a real conversation.

Again, no different than having an Ivy League education. There are big advantages to scope/reach/opportunities in the industry.

At one point in my thesis defense, I derived several equations I hadn't seen before, on the fly, such as "What is the time resolved fluorescence of a fluorophore in 4-dimensional space?" and finally understanding ergodicity (https://en.wikipedia.org/wiki/Ergodicity).

My defense wasn't about determining if I was an expert and qualified to write a dissertation in my field (my questioners already knew that), but to determine if I was a well-rounded general intelligence capable of out-of-task prediction.

In my phd experience in the USA we had oral qualifying examinations which involved whiteboard derivations. The thesis defense after writing was really mostly focused on probing the results in the thesis.

Depending on your definition of success, I'd say determined to succeed could well be the same thing as obidience.