Coding is more like weight lifting then running. There are some weights that some people just can't pick up. And so if the job involves lifting things that are usually light but occasionally heavy, then a squat becomes a good measure for candidate.
But once everyone knows a big squat is the way in, they train squats, they show up in lift suits and the whole thing stops being a good measure.
Because on the job you don't squat in a squat rack, but pick things up where they are found, in the conditions on the ground.
Now you have a bunch of people optimized for a task that isn't exactly the job.
then you try to get them to explain TCP/IP or fix a for-loop and they're stumped.
We had recruiters tell us not to do tech interviews, nobody likes them, etc, but our team was so strong for it.
And really, we were mostly testing generics. Use stack overflow by all means, but just know what you are copying and how to modify it!
Meanwhile, I've helped several clients hire new remote devs before and all it took was a few basic coding questions to weed out 90% of the candidates, followed by a small paid trial task before hiring them long-term. The results we got were fantastic, completely merit-based, and easy to reproduce.
1. A lot of people have no idea how to evaluate a candidate, even with a rubrik. I have been in many evaluations where I tracked with particular interviewers said about a pros or cons of a candidate and it is near noise when you get beyond candidates that are clearly not qualified. Even simple questions like "dedup a list of integers". Sometimes the interviewer really focuses on if they maintained order of the dedup list, other times it doesn't matter, other times they care about time complexity, sometimes not. This is especially bad when an interviewer is allowed to choose the question that they ask, because they choose questions that have something they find "cute" and if the candidate doesn't pick up on that "cuteness" it's really hard to evaluate them.
2. A lot of people I have met think that an interview process should be able to evaluate some random developer you pluck off the street without prep. The larger companies are less inclined to believe this but smaller ones I have found either don't know or don't have the person-power to prep interviewees. Again, a lot of people think whatever knowledge they like is the knowledge everyone should have.
3. A lot of people I have met think that you can understand after 5-7 hours worth of interviewing, where one person probably only interacts with the candidate for an hour or so under stressful conditions, it that translate to someone you can be around for 40 hours. It just doesn't.
4. A lot of companies look at interviewing as finding the missing puzzle piece of the company rather than shaping the puzzle piece. They just want candidates to join and go. That's really tough to do at even a moderate scale.
I think a lot of companies don't really realize how both difficult and high leverage hiring is. You're spending a few hours with someone to determine how they will work over thousands of hours. They want hiring to be easy, you just see if the person passes the checkmarks and they are in! And if you don't confront the reality you can't adequately solve the problem.
Maybe this is objectively The Right Way to hire developers, but good luck convincing the accountants of that, and good luck convincing your current developers to pay for it.
Note: I’m not the original commenter and I have not used this hiring method, but I do have to budget.
Are you applying for that job? Are you willingly taking a lower salary because they're paying the people you interviewed against hundreds or thousands of dollars?
That kind of sounds like a general organizational dysfunction: someone designs and elaborate and expensive process that doesn't work and rejects 100% of the candidates (which was almost certainly sold a a "process improvement"), then people end up working around it to get things done.
Programming is more of a trade and academic institutions barely teach the craft.
OPs point is that if you were a hospital, you would take a different stance on the relevance of a doctorate degree to your hiring criteria.
For doctors and lawyers one may need to go a long mile to get necessary paperwork done in order to work in another country
For example, INCOSE for systems engineering. Ladder is based around 3 career steps: book knowledge; ~5 years experience; senior roles on major systems. Qualification depends on documented experience, reference systems, review from within the community, and interview.
Couple of solutions:
- Open source contributions, especially in the tech you are hiring for. Need someone who is good with spring boot? Makes sense to see the open source contributors in the spring ecosystem.
- Open sourced side projects. You get to see the code the candidate wrote, even before hand, and have them walk you through the decisions they made.
White board is still path of least resistant - its for the lazy hiring managers, and they usually do not even know how to do it well which is why there is always so much attrition.
Edit for clarification - open source + projects do not have to replace the in-person white board. Rather, it can be additive.
Who would you rather hire - someone with hi-quility open source contributions, or someone who can implement LRU cache bc they happened to do that problem on leetcode?
The places I've worked in the past 15 years generally take open source contributions as a very slight positive signal, but only one of many.
What's a strong signal in your experience?
Plus, nervousness and comfort with the environment and questions makes the error bars bigger.
Depending on the level, but assuming a Senior role, then some stronger (not strong, just stronger) signals are:
- ability to code basic solutions in the language of choice - ability to reason about code complexity
- ability to get to a solution
- ability to explain their reasoning
- demonstrate they understand the nuance and complexity of software development
- understanding common tools in the domain they are interviewing for (sharding for distributed systems for instance)
Same, but on the negative/bad side of things:
- arrogance, argumentative, rude (nervousness is totally fine and expected)
- one true way-ism
- complaining about interviews during an interview
- inability to solve basic coding questions
- lack of understanding about basic data structures (too many people I've interviewed don't even understand how a hashmap is used)
Like, whiteboard interviews and leetcode grinding are not good filters, no argument there. And OSS is good and we should encourage more contributions in general. But using OSS/hobby contributions as an interview filter is just as bad as leetcode, in that it inequitably narrows the field of candidates and further compounds the hiring diversity problems in the US tech industry.
I can see a lot of value in contributing to open source being an alternative way of demonstrating competency. However, I don't think I'd want to see it become the only or primary way.
It doesn't have to be a replacement for traditional interviews. Rather it could be a way of distinguishing candidates instead of only relying on a white board.
When I was trying to get my first job out of bootcamp, I had a built a mobile app with my team mates, and we had it on our phones. The idea behind that was we could pull put our phones during interviews and show our interviewers exactly what we did.
"Btw, hiring manager, I happen to have that app on the projects section of my resume on my iphone as we speak - would you like to see it? Happy to walk you through the commits I made on github if you would like."
But...
Most hiring managers were not interested at all. Instead, it was all white board. The whiteboard brain worm infection in hiring managers just runs too deep, there's probably no cure at this point.
Who would you rather hire -- some one who could show you an actual project, code and all, or someone that can implement LRU cache in under 25 mins bc they happened to already do the problem on leetcode?
I help maintaining a quite popular package ATM, I prefer when people do it for pragmatic reasons or for passion, rather than to improve their GitHub stats :/
There is also now an underground economy built around fake open source contributions and even entire projects. This exists precisely because some employers do care about open source contributions, so hiring managers now have to verify that those contributions are real.
Yet we've gone through many iterations of the interview process (and testing, things like leetcode) to avoid having to basically accept a gut check of "do I think this person will perform well".
It's more of an art versus science than I think people give it credit for.
So is interviewing itself.
Even in academia, I think this view is counterproductive. We shouldn't be imposing suffering on folks just because that's how it's always been done. I prefer to take a different approach: give people every chance to make themselves look good, and see if they're able to capitalize on the opportunity.
I think "job talks" could have a useful place in industry hiring. Ask interviewees to prepare a 5-minute lightning talk about any technical (or technical-adjacent) topic they want, and open it up for Q&A from the panel afterwards. This is a way both for the interviewee to set the stage on a topic they're confident in, and for the interviewers to gauge the applicant's communication skills and technical knowledge.
Ideally, entry level positions would hire for potential with the expectation that they will be training their employees.
I have a PhD on Embedded device security and last time I went through 3 interviews with HR people before I ever got to speak to an engineer. 90% bullshit questions.
However, I can not imagine a situation that would ever call for 2 or more meetings. Nor can I visualize any circumstance where a meeting with HR would help with highly specialized position hiring.
Once your job description calls for 5+ years of experience, a master's or even a PhD, maybe a certificate or two and even publications on the field, there should not be more than a couple hundred applicants. Not even a hundred to be honest. And every one of those claims can be verified by manual online searches or automated tools. Furthermore, the only way of verifying the applicants actually know about those subjects is a meeting with a fellow engineer.
>And every one of those claims can be verified by manual online searches or automated tools.
No it can't. There is no central registry of every graduate in the world, and even if there was it would be expensive to have a subscription. If it was cheap to do background checks on everyone, it would be done up front instead of after they decide to hire you.
Also, leetcode is just one way of doing technical interviews. There are others that also do not require going to Stanford.
I blame it on the very unspecific headcount demands in software projects: there's never a hard target number "we need ten keyboards to be manned at all times!", life just goes on when you are nominally understaffed. Perhaps slower, perhaps with more crunch, but nobody really knows what difference a hire or two would have actually made. So software development HR keeps throwing out their net, who knows, perhaps tomorrow the perfect candidate applies, and rejects a lot. It's a bit like the pattern of attractive people trapping themselves in an infinite dating treadmill, but with the plot twist that their salary depends on it.
IMO this doesn't make sense. I don't think a senior dev should have problems switching stacks or languages.
From the buyer side it makes a lot of sense, the problem is that when you're on the selling side and too old to get accepted in a junior role, you either have the precisely fitting CV (and can prove it through the interview stages), or you're bound for a tech stack consisting of an uber-accepted car.
Most airlines do the same. The majority of pilots flying 737s were hired without that specific type rating. Usually they're moving up from flying smaller jets at regional airlines.
For all the talk about hiring the best, once they're in the goal becomes to stop hands-on development as quickly as possible and to add friction to improving the knowledge repository as much as possible.
I feel like the financial engineers are eating the lunches of CS folks whose only skills is to "sling code". These 22 year old finance kids didn't need a CS degree (or at least major) to develop the coding skills necessary to express themselves and every, single one of them is highly competitive. I know one CS guy who is doing well too, but he's an industrial engineering double major. Definitely hearing a lot of moaning from the CS kids that just followed the curriculum and didn't develop some side hustle already.
Point is that we've very quickly hit a point where the market is saturated for people who can only code and nothing else, at least from my observation in the US mountain west region. School projects and even internships don't seem to be cutting it anymore, at least if you're attending a state school I guess.
Ten years ago if you could write good code you could get a job. Yeah you might not get a FAANG interview if you didn't know anybody and you went to Southern Regional State College or whatever, but you could get a job making twice what your English major friend got, and a lot easier.
Being able to write code is no longer a differentiator. You need to know something about product, or design, or finance, or entrepreneurship, or at least show through the other parts of your resume that you can be a benefit in some of those areas given the opportunity.
The root is that you are not owed a job at all, let a lone a cushy six-figure one where you can type JavaScript into a computer for a couple hours then call it a day. So if you believed the lie of "just go to school for 4 years, major in CS, do what your professors tell you to, and you'll be set for life" then you're in for a rude awakening when September rolls around and you're still looking, or you're writing Java for a bank for $45k/yr.
I was 100% this guy 10 years ago. Engineering school dropout but could build things on a LAMP stack, so I had more work than I could really handle. But it should have been apparent to anyone that this couldn't last, so I went back to school and explicitly avoided software because if I could teach myself this stuff, these econ, finance, and stats people were going to as well. And here we are!
> writing Java for a bank for $45k/yr
Yep that's the type of job you're going to find in Pocatello, Idaho... working for Regional Bank of the West integrating bank software their mobile apps and websites. The interesting jobs are currently going to the mechanical/electrical and hell industrial engineers who did a code bootcamp. This isn't my suspicion, it's what is actually happening on the ground and I hope the 19 year olds feeling a bit of buyer's remorse on their choice of degree program take note.
When consultants come in and ask for data, this is the person that gets them what they need. This is also the person that stays off the chopping block whenever the consultants propose their solution.
*)Easier as in the recruiters will actually reply back or even them hitting you up instead of you getting ghosted when you apply, not that they make the interview process much easier for you, you still have to pass the same bar as everyone else but you have a much greater chance of being called back
Of course that is for algorithm interviews, in less objective interview formats you are right where you studied etc is a massive factor.
I don't think this is altogether a negative, if you're hiring an investment banker then yeah the rich or connected family they come with is a benefit so you're probably in a good space regardless.
No doubt, software developers without degrees have been very successful in borrowing money. And burning through it.
It's because tech dev / coding jobs are one of the few white collar fields that, historically, have not de-facto required a pile of advanced degrees. Maybe i'm getting old, but "you don't need a degree if you're hot shit" has long been part of the atmosphere: from old school hackers to the long history of founders who drop out of school to work on their startup, etc.
This directly leads to interviews being the gate: if we can't know what you are by what degrees you have / what papers you published / what lab you did your dissertation in / what bar exam you passed / what hospital you did your residency in / who you clerked for / etc. ... then we're going to have to do all 4-8 years of post-undergrad pain and suffering all at once, during the interview, so grab a whiteboard and buckle up!
It's ... less than ideal.
edit: what's less than ideal is the perverse incentive force that creates bad interviews. The recognition that a degree doesn't guarantee job fit is not a bad one, though; you really can be more successful in software dev if you're self-taught / uncredentialed than you can in many other disciplines.