"That guy is our best player! He scores more points than anyone else in the team by far! Quick! Let's pull him out of the game and make him the coach!"
Why not keep him in the game and just look for someone who can be a good coach?
If you have a high performing dev who wants to be a team lead and you hire someone else for that role, you risk demoralizing them and causing them to look elsewhere.
Morale is important. Promotions, real promotions with more responsibility, are important
It doesn't always make sense to compare two occupations with each other and pretend they can use the same approach for hiring people.
Piloting specifically requires a qualification like being a doctor etc. These qualifications are recognized and remove the need for assessment of a skill during an interview.
However there is no standard international qualification for software development like these other professions that are restricted -- you simply can't call yourself a doctor as these professions are regulated. Anyone can call themselves a software developer. Do we really want to regulate the software dev profession?
What devs and mgrs do is less machine-like, and it follows that what works for "making sure you have pilots who can fly" doesn't quite translate to "making sure that you have managers who can manage".
Actuaries, accountants, doctors, lawyers, even hair stylists all have qualifications that are required to do the job that says they can perform the basic tasks safely. What's so special about ours that we can't do the same? I hate needing to prove that I can fizzbuzz to each company individually.
If some hiring manager or interviewer scores badly, well, he may end up not interviewing anymore.
We want even our non-hires to have a good experience interviewing with us.
I learned that I don't want to get better at management, so I am no longer a manager.
I assume that managers have ways to evaluate managers, but I don't know the gruesome details.
What insults me is when someone wants to be paid more than a doctor based on no qualifications and the fact that they tell me they are worth it. If you cannot do the exam but can discuss it with the company who accept your argument then great, if you can't then you probably don't want to work for them anyway so just chalk it up to experience and move on.
What's stopping you from selectively removing exams from people who actually do have the qualifications?
> If you cannot do the exam but can discuss it with the company who accept your argument then great
I could flip it around and say, if you cannot hire strong engineers without exams, discuss with your company to make someone else the hiring manager.
If I was your employee and I saw positions remain open for years (burdening existing employees), I would question your ability to set good processes.
Why is your time more valuable than mine? Why should I invest my personal time in to your hiring process when you can't bother to spend the same amount of your on the job time, to hire someone whos going to add far more value to your company than your company is going to add to their profile?
No one is forcing anyone to apply for a job at any company. As the seller of labor, you get to choose how much to invest in any one opportunity. Just like someone selling apples can choose to open a roadside stand, try to get into a grocery store, or sell direct to consumer.
The buyer, of course, gets to choose the process by which they evaluate the labor (or the apples). If you don't like how the buyer evaluates what you are selling, find another buyer.
I would much rather take a short coding exam.
It might be more common now in the remote era but this has been a standard practice for years so I don't think it's a valid excuse.
There's a couple of "problems with this idea in reality" I feel I should point out:
* The process of firing and hiring someone is often a *very* bureaucratic thing that requires many people to sign off on it. It can often take weeks or *months*. That's a lot of wasted time when you actually need someone to fill a role.
* Some countries just plain don't let you do this (e.g. France). Once you hire someone you're basically stuck with them unless you can prove they did something really, really bad.
* It can be downright evil to just up and fire people shortly after hiring them just because they're not as efficient as you'd hoped. You'd *think* that having a process like you suggest would only get rid of people that lied on their resume (good riddance) but in reality it'll be abused by managers to shape their reports in ways that are beneficial to them but not necessarily the business overall (zero-consequence firing).
* There's something to be said for letting people through your hiring process that aren't qualified... It's downright dangerous from an operational stability and information security perspective.
I’ve seen this happen plenty enough even in SV. If someone wants you fired - they’ll do it asap.
So even if it surfaced the best candidates, hiring anyone that applied and then firing 90% of them after a week would result in a depressing work environment and an ever shrinking pool of qualified candidates (who wants to upend their life to take a job they only have a 10% chance of keeping?). Not to mention the cost of onboarding.
This is how we get to the status quo: do as much filtering as you can before you hire.
Slacking off during the hiring process with the intention of firing a large portion of the people you hire is a violation of the social contract and a huge asshole move.
HR is really only qualified to filter out based on the qualifications on paper. These are sometimes misleading and occasionally outright fabrications. Those can really only be verified by someone with the appropriate technical skill to separate bullshit from an accurate representation of the abilities of the candidate.
If the hiring manager wants the firehose of the "everyone who clicked quick apply" on LinkedIn or Indeed, then fair 'nuff. However, my reading of the comment that I replied to was the issue that HR wasn't filtering out unqualified applicants. Removing HR from the hiring process won't improve on that issue.
Much better to get someone qualified in the first place.
I typically use a small coding test, but not even as elaborate as described in this post — more like a 10 or 15 minute discussion/problem just to see if they really do know what std::string is or how to use an op amp or whatever. Some people can really bullshit well. Consider this the simple “do they know which end of the knife to grab when picking one up?” test.
Once that is clear, a conversation (usually just a conversation about a hypothetical problem) works well.
I don’t understand the coding tests that test your tools knowledge btw. If you know how to think and are somewhat familiar with the appropriate language then you can pick up the specific tools and frameworks. We hire for the long term so said tools and frameworks will change anyway.
Sadly, I think more and more companies are posting for jobs/positions they don't really have available/open at the moment and are just proactively trying to find potential workers (because this whole hiring process is a mess and take so long).
Hiring is becoming an always on activity.
I can offer this bit of perspective on the practice, without justifying it because I do think it's bad. We (again, engineers) spent a long time pushing for hiring to move away from the desperate search because we need someone immediately model to keep our lanes open for whenever someone awesome is looking for a job. Management finally bought into the idea...kinda. Basically, what we're doing is the mixed up interpretation of a that idea, after it passed through the bureaucracy. Instead of keeping just a line open, we're still trawling with big nets, scooping up way too many resumes and letting those people down.
The root of the problem as far as I can see is that screening is atrocious. The people at the front end of the process, the actual recruiters, have no idea what we are looking for, and the output may as well be random. I still don't know how to fix it. :/
One is to have the hiring manager or someone who would be working alongside the new hire write the job description and not allow the recruiter to write it (no “17 years of experience in Airflow”).
Another is to have an engineer do a phone screen after the recruiter has screened them. Having the recruiter listen in is good as they will hopefully learn more about what is wanted.
Another is to carefully set the recruiter comp to take into account not only hires but percent that fail a phone screen.
Your recruiters aren’t your enemy but they are recruiting, not programming (you probably couldn’t do their job either) so there will always be a mismatch to some degree.
There are also people interviewing just for practice or to find a way to negotiate a raise with their management.
Of the people I have interviewed in the UK for "Senior .Net Developer", I would say that 75% did not meet the grade and that was with face-to-face interviews that took a lot of my time. An exam isn't perfect but if someone can't answer a reasonable spread of questions to a half-decent degree, then I don't want to hire them and it saves me hours of my time. Sorry.
Odds are, you're getting not good enough because you're using programming puzzles. Instead, why not quiz them on the work that they'll be doing, so you know that they can do it?
Firing people is not as easy as you think in many places. Also, managers are human. Accusing someone of lying and going through the termination process is mentally taxing!
If a simple screen takes twenty minutes of time, it’s rational to do. If people get offended and aren’t empathetic to why you’d do this, would you really want to be on a team with them?
I agree it's a complete snowball! It's absurd :) And, it's reality- this is how litigation & workplace regulation in America works. For example depositions are so crazy, not only can you be deposed as a party in a lawsuit, you could be deposed for extra information even if you're not the one being sued (i.e. you're in a related industry as someone in a lawsuit, so the court compels you to a deposition just to talk about the industry, what's standard there, etc.)
* Pay off any accrued time off
* COBRA insurance
* Unemployment
* Opening up the hiring pipeline (ads on job boards, recruiter, flights for potential candidates, training for new new employee)
* Potentially pulling other employees to fill the role while hiring again (Overtime and stress to existing employees)
Versus
Sending an email and saying no.
This is not that simple. Your hiring process is a funnel with certain conversion rates at wach step. If you overcomplicate it, your hiring timeline extends, increasing time to market for your projects and having some impact on the revenue. If you oversimplify it, risk of bad hire increases, but it may still worth it, if funnel gets much better.
Firing also requires a fair amount of person-hours to do, and if you fire someone and need to replace them those costs stack.
By having actually done it, and understanding what processes are involved.
.
> I do not think this is correct estimate
It is.
If someone successfully works shipping code but can't reverse a linked list, does that mean they're lying about being able to code? I've seen way to many jobs that test for expert level skills for a job that is just to add small enhancements.
On a whiteboard? Probably not.
In front of a computer, with a compiler and manpages handy, being able to test and debug it right away? Absolutely, yes.
No, I think it means they didn't prepare. But, short of examining git commits, how can you know that someone successfully "shipped code"? References, maybe (though that is harder for folks who may have fewer references from past jobs). If it is open source, sure. But how else?
> I've seen way to many jobs that test for expert level skills for a job that is just to add small enhancements.
Yes, I agree! That is a bigger issue, where the job desc/interview are disjoint from the actual work. That's a red flag. It's hard, as a hiring manager, though, to keep the job desc real. I find that I sometimes throw wish list items in ("knows <tech> at an intermediate level") because the alternative ("can learn <tech>") is so hard to test for. But as a candidate I find it extremely aggravating.
I don't know how to square that circle. If you figure it out, please let me know.
My favorite kind of interview (on both sides of the table) is where I can do the work that is typical, using the resources that would be available on the job.
Obviously yes? That's a basic assignment for a data structure class given to freshmen and sophomores. One would expect that a "senior" developer (degreed or self-taught) would be better than a college sophomore. At the FAANG level, it would be a phone screen question and a softball one at that.
As it turns out, designing a fast algorithm involves a lot more than what is covered in the textbooks. This is why something like Timsort can generally outperform quicksort even though a naive algorithms course overview may well come to the opposite conclusion.
If my job is not to design these algorithms, why does it matter if I can write a naive implementation of one?
I mean, it's true that there's a lot of code that really is at the FizzBuzz level:
switch (c) {
case 'h': case 'D': menu.left(); break;
case 'j': case 'B': menu.down(); break;
case 'k': case 'A': menu.up(); break;
case 'l': case 'C': menu.right(); break;
default: /* no op */ break;
}
def find(self, name):
for thing in self.contents:
if thing.is_a(name):
return thing
vaders.hits(bullets, (vader, bullet) => {
blam(vader, 2.5)
score += vader.bounty
vader.die()
bullet.die()
})
if (vaders.where(({y}) => y > 400).count()) startGame(120)
if (!vaders.count()) startLevel()
async def take_token(self):
self.update()
while not self.tokens:
await asyncio.sleep(self.recharge / 10)
self.update()
self.tokens -= 1
// Animate explosions.
poof.move()
poof.each(p => {if (--p.ttl < 0) p.die()})
// Chain-reaction explosions.
poof.about_every(64).each(p => {
if (poof.count() < 1024) blam(p, p.w/12)
})
// Limit number of explosion objects to guarantee responsivity
if (poof.count() > 1024) poof.about_every(2).die()
Reversing a linked list is just about the simplest task that requires that kind of reasoning, and it has the advantage (or disadvantage) that understanding the puzzle doesn't require, or benefit from, any problem-domain knowledge. You could substitute binary search or insertion sort or something. Another example I got, which takes a few more lines of code, was "implement an integer stack ADT which can always tell you the maximum number on the stack in constant time".
I think the problem is that there are over a dozen different kinds of "seniority" but we collapse them all into one term. I got so frustrated I wrote a blog post about it: https://www.mooreds.com/wordpress/archives/2812
Also there are ancillary skills that make the problem much less difficult, so if you can't solve it, you don't have them. Joel Spolsky famously said that nobody can reverse a linked list without drawing diagrams on paper, and that's probably true if you're doing it imperatively in C. But it's in the category of "problems that become much simpler with functional styles of programming." The following took me 5 minutes, most of which was struggling with the syntax of OCaml that I'd forgotten:
$ ocaml
OCaml version 4.08.1
# type 'a list = Cons of ('a * 'a list) | Nil;;
type 'a list = Cons of ('a * 'a list) | Nil
# Cons(3, Cons(41, Nil));;
- : int list = Cons (3, Cons (41, Nil))
# let reverse lst = let rec rev xs ys = match xs with Nil -> ys | Cons(x, xs') -> rev xs' (Cons(x, ys)) in rev lst Nil ;;
val reverse : 'a list -> 'a list = <fun>
# reverse(Cons(3, Cons(41, Nil)));;
- : int list = Cons (41, Cons (3, Nil))
$ ocaml
OCaml version 4.08.1
# type 'a list = Cons of ('a * 'a list) | Nil;;
type 'a list = Cons of ('a * 'a list) | Nil
# Cons 3 (Cons 41 Nil) ;;
Error: Syntax error
# Cons(3, Cons(41, Nil));;
- : int list = Cons (3, Cons (41, Nil))
# let reverse lst = let rec rev xs ys = match xs with Nil -> ys | Cons(x, xs') -> rev xs' Cons(x, ys) ;;
Error: Syntax error
# let reverse lst = let rec rev xs ys = match xs with Nil -> ys | Cons(x, xs') -> rev xs' Cons(x, ys) in rev lst Nil ;;
Error: The constructor Cons expects 1 argument(s),
but is applied here to 0 argument(s)
# let reverse lst = let rec rev xs ys = match xs with Nil -> ys | Cons(x, xs') -> rev xs' (Cons(x, ys)) in rev lst Nil ;;
val reverse : 'a list -> 'a list = <fun>
# reverse Cons(3, Cons(41, Nil));;
Error: This function has type 'a list -> 'a list
It is applied to too many arguments; maybe you forgot a `;'.
# reverse(Cons(3, Cons(41, Nil)));;
- : int list = Cons (41, Cons (3, Nil))
#
Similarly, there are problems that become much easier with other paradigms of programming, such as backtracking search.
Anyways it is a bad question to ask because it is very easy and is frequently studied so candidates just memorise the answer.
I agree Leetcode questions are generally not a good signal because they're often removed from everyday problems one may encounter. Though they may be fun to do. If I could get a job just solving Leetcode problems, I might chose it over refactoring messy codebases or joining disparate APIs.
And from my limited experience on the hiring side, while a trusted referral is worth much more than a programming test, a good report from a potentially adversarially chosen reference is worth much, much less. It's much easier for an unscrupulous person to find a friend who's willing to inflate their achievements on the phone than to provide on-the-spot solutions to programming problems they lack the skill to solve themselves.
A good hiring manager will be able to sniff that out rather quickly. Following up on references doesn't mean calling it a day after one random phone call either.
Source: I am FAANG ATM and the new grads I work with outperform 10+ year tenured engineers at other companies I have worked at.
It is! Where I work we have a min of three references required for non-entry level jobs and it reveals quite a bit at higher levels. It does require more effort on the part of the hiring manager though and I suspect most would rather punt it off to a test than do the due diligence when it comes down to it.
I've walked out on interviews that demanded this sort of homework before. I told them 'professionally' that their idea of demanding 4h of my time after 3 rounds was farcical and insulting.
Ditch the homework completely, or tell about the homework up front. Compensate - I don't work for free. And if you want a fair assessment, put me in contact with 1 of your engineers or devs and lets talk out a problem. Or review my body of work in that thing called a "resume".
If you have multiple bad hires that lied about skills they didn't have, then perhaps you ought to reconsider who is doing the hiring decisions. That's been a big one in previous orgs I've worked in. The hiring group was static, upper managerial level, and didn't understand the problem domains of the positions they were hiring for.
1. We tell candidates about the homework during the phone screen which is about 20 minutes, if they want to duck after that no time wasted. No one ever has.
2. We don't compensate for it since it's the 3rd interview. The idea that I can take 4 hours of your time for a half-day whiteboard interview but not a "do a simple task that should take 30 minutes at your leisure" is weird. We've never had anyone have an issue with it because we have the one of the least intensive interview processes in the area.
3. People lie or vastly overestimate their skills on the resume, surprise. If you submit code samples or your Github is good enough then we skip the homework. Designers/artists have it right with the "portfolio" -- resumes are very nearly worthless. We don't demand code samples or FOSS projects because we don't want to assume people code after hours and obviously can't share code they wrote at work but in that case we need an alternative.
3. You are interviewed by the team you would be working with. There is no hiring group. You have contact with all the engineers. You can even talk over the assignment with the team at the end of the second interview.
It's really not that weird at all. If you give me 4 hours of face-to-face time 1:1 with one of your engineers, I have a guarantee that you value my time as highly as you value your engineers'.
If you give me 30 mins of "homework", I have no guarantee at all, that you don't have 100 other candidates in this stage of the interview process, and one of your engineers is going to dismiss my application after 5 mins of looking at what I've produced and give me no chance of responding to any judgment they are making that I disagree with. For example I once got "I don't like the fact that you use semicolons in Python where they're optional". If someone thinks that's a valid reason not to hire me, I want the right to face my accuser.
Many of us have been through terrible hiring practices. And we're sharing them here. And when you give examples or critiques, we're critiquing them right back.
I've been around the block. I've done lots of interviews in tech. So have others here (naturally!). I've noped out of many for huge semaphore-sized red flags.
But the point stands - I'm not going to do "homework" without getting paid for it. I would much rather a sit-down with a fellow engineer and talk out problems. That shows the company is also willing to put time in as well.
Sorry for being incendiary, but this hits me as a recent trend that is nothing short of insane. I went to a top-ranked university on the highest-paid scholarship I was eligible for, got a Ph. D. there, then got the job that all my peers wanted but only a handful actually got. Then I booked 10 years of industry experience. -- You wouldn't believe the life sacrifices I've made to be able to have all of that on my resume. If someone tells me that's all "worthless" and they'd rather ask some stupid puzzle questions and see some indication of how passionate I am about working for some no-name company, or whatever the latest hype is, then that's simply where the interview ends for me.
A typical resume has none of that and is more like:
. BS from some un-ranked university
. 2 years at some company you've never heard of
.. Did things you have no way of verifying
. 2 years at another company you've never heard of
.. Did more things you have no way of verifying
I once had an interview with a scientist turned to industry who couldn't demonstrate how to load a text file after 30 minutes of trying. I mean, what should I have done with him? The position required lots of algorithms and data processing.
The fact is that there is such a thing as an "impressive resume". There seem to be people out there eager to dismiss the value of those. Frequently an explanation will be that they themselves don't have one and are engaging in cognitive dissonance reduction. In fact, one can say more generally that psychological forces will act powerfully in such a way as to make hiring managers want to hire people who are just like them, even when they themselves are worse than the best candidates they could hire.
That's the reason why having an analytically well-reflected hiring process, instead of just hiring who you like, is important, and resumes have an important role to play in that.
Nah, I'm all set.
Haven't you read the parent post? Most people who claimed to be senior were "scary bad". In my interviewing experience it was the same - one qualified candidate in 10.
Until there is something quite objective about measuring performance against some kind of qualifications, then it is in danger of being more to do with personality than genuine value.
Former hiring manager.
It's unfortunate because it is a bit insulting. However, there are some people who lie. They lie about being able to code. I don't want to hire a senior dev who I then have to fire shortly thereafter.
A simple, short coding test that any senior dev should be able to do quickly in their language of choice or a longer project that they get paid for are my two favorite options to filter out folks who say they are senior devs but can't code.