The crazy thing was that over half of the applicants that passed a resume screen and phone interview could not complete the homework assignment. Over half. Some couldn't get it at all (either gave up or had compile errors), some got it to run but it didn't work, a few got it to mostly work but their programs had serious bugs, and the last few actually got it right.
The assignment was to read a file containing a list of numbers (some formatted incorrectly, so there was some very simple parsing logic involved), call an API using each correctly formatted number as a parameter, and store what the API returned to a file.
I'm still shocked over half of developers couldn't accomplish that task. Makes me wonder what they were accomplishing in their existing roles.
I'm perfectly competent at writing complex code. In fact I'm usually the guy who fixes bugs or perf issues that nobody else believes exist.
A problem you came up with on your own is going to seem simpler to you than to everyone else. Why? Because you thought you picked it at random but your subconscious picked the one it thought was good. In other words, you picked the one you were already primed to answer, and it seemed simple.
So either people are going to work at it longer, the timid will give up (there's nothing wrong with timid coworkers as long as they have someone to listen to them), and the brash will get it done in the time allotment by cutting corners.
Are those the people you were trying to hire? Because that's who you're gonna get.
I absolutely would not hire someone who failed that test. Your defence of it seems bizarre to me.
I think these tests are a great tool, but like anything we all just need to be aware and careful of the biases introduced. Also, we need to be mindful that really good developers who are employed, have families etc, may not have an over abundance of time to spend on these, so if 10 companies each send a challenge that takes 4 hours, and it's unclear if the assessment will lead to further interviews, some better candidates may give up on the process or be selective about the ones they complete.
I think this creates an additional bias, that the people who complete your assessments may be the ones with the most time to invest in the process (ie the currently unemployed).
The catch? There is no internet in Cuba, so they had to do it using a book, and asking questions to a programmer who was helping them there.
It's also better to aim small (a la FizzBuzz) and walk up the complexity slowly, as opposed to starting with "Solve this ticket from our prod JIRA queue, with tests please" (which I have actually been given as a test!)
I like to aim for a very simple task, like proposed above, with maybe one non-obvious part or edge case (which the spec mentions, so it's not a trick question). Just enough to demonstrate a) clean coding for the simple parts, and b) ability to solve something without using a completely naive solution.
Uhuh.
I had a shared library running on a bunch of systems. One was VxWorks. Cross compiled. I knew nothing about VxWorks except that the code needed to run about 8x faster. I got about 2.5x out of my code by improving locality of reference problems that appeared to be intrinsic but just required experience to see. For the rest I filed a bug report to WindRiver and five to the cross compiler writer. At the end of this process I knew more about the VxWorks administration than anybody else in the group.
People give up, even when there's a clear business case for continuing.
But reading a file, doing some parsing adjustments, and calling an API sounds like a small amount of python code... so I guess not so bad :\
for num in $(egrep --only-matching '^[0-9]+' input-file); do curl -vv -X GET "http://example.com/api/v1/foo?num=${num}" -o "resp-${num}"; done
edit: assuming "parameter" means query paramYour one liner is neat and all, but now you have to do 2-3 hours of initial work + w/ever time it takes for adjustments in 2 hours scheduled for an interview.
You could've thought out this toy architecture up front with initial simple implementations or at least discuss what they expect you to provide, but instead you wrote a shell script for some nefarious reason, even though you saw obvious disconnect between estimated time and time you actually spent.
When I was a junior developer, I was just trying to get code to work. When I learned design patterns, and had some bad experiences, then I started architect "robust" architectures for everything. After perhaps 6 or 7 years of doing this, I started to question my insanity. Sure there were several instances where I was like "Damn, really glad I built it like that" but there were an equal amount where it just ended up turning a simple problem into a complex problem. There's no good excuse for that.
So now, I go high school simple first, and college hard later. If it means I have to do some rework, fine. I'm okay with a little rework, because in the end it aggregates to a simpler platform that took less time to build.
Anything beyond 10-20 lines for the solution to this problem is pointless levels of abstraction and would be a huge red flag.
$ cat file
1
1.2
3
4.5
$ egrep --only-matching '^[0-9]+' file
1
1
3
Surely still demonstrates ability, but an automated testing suite is going to throw it out.Never had a problem with that. It's well known that C++ can't do anything on it's own.
We have an in office test--the dev sits down at a laptop with internet access, gets clear instructions and scaffolding code. We know that the tasks take n minutes because we see the time that candidates take.
There is no incentive to cheat. Why would you want a job you are not qualified to do.
From your experience, it sounds like these are good tests, because it's screening for qualified candidates, unlike the phone interview.
money.
I'm not commenting on whether that's a good or bad thing, because good or bad it's simply the way it is.
What if you joined Google and was poached by Facebook after 1 year?
This does not contradict the fact that it still leaves/can leave a bad impression on impression if your were willing to be poached without a very good reason, since can be considered as a strong sign that your are not loyal to your employer and only hunting for money.
2. Poaching is rather uncommon in Germany. As I outlined in 1 being willing to be poached multiple times is a sign that you are not very loyal to your employer. Thus there is a strongly reduced incentive for employees to be poached.
Also for poaching to work, you have to offer lots of money - I'd be really surprised if there are lots of employers that are willing to start such an "arms race with money". Why not simply offer decent money in your regular job advertisements and get good people without any arms race?
Poaching might make sense if there you want to get "rockstars" that are not available on the "free job market", but can only be poached. But I'm also not aware of any industry sector in Germany where there exist employees that are treated as rockstars. :-(
Just imagine a random company in a tech hub. They host a meetup once... the days after the meetup, most of their employees have been contacted by recruiters and other companies who attended.
If they want anyone in particular and/or if any employee's salary is not on-par with their value, here they go! :D
I am not so good with words sometimes... Haha...
Why do they do it? I have no idea.
I once asked a candidate to write a for loop in C#[0], using my laptop and Visual Studio and watched for a solid 5 minutes while he struggled -- And I don't mean jitters around syntax, I mean, failing to declare a variable properly and while he knew what a for loop was, he had no idea how to write one in C#. His resume touted a series of projects that we estimated put him a little above a Junior developer and the only language he referenced was C#. I don't normally do these kinds of "on the spot tests" because I, personally, hate them, but I was cheering when I read Coding Horror's "Fizz Buzz" article[1] on the subject.
This was the most foul example and my only experience is with developer and ops/security positions, but if you're getting half of developers who are in the same time zone as the title[2], you're doing well. Does this happen in other parts of the business world? Some of the folks we get in would be like a Walmart cashier applying to run a finance department because "they've handled money before". And what if they got the job, do they think they'd come close to being successful? I'm all for the "fake it until you make it" and I'm very willing to overlook weaknesses around experience/education if the candidate is really passionate and I can get enough of a comfort level that they'll be a quick study and love the job, but you have to be realistic about your deficiencies.
A lot of the problem, I've found (being on both sides of this in the past) is with recruiters, as well. They tend to operate on a "throw darts at the wall, blindfolded" approach, with the assumption that if they send out a million candidates for a million jobs without regard for qualifications, that they'll get lucky, one will land somewhere and they'll get a commission from sheer volume on bad odds. This serves nobody well -- it causes companies to put up multi-step gates that involve people who aren't close enough to the field to discern a good candidate from a bad one via resume performing a filter operation that disproportionately disqualifies the really passionate but "not quite a perfect fit on the qualifications side" and it causes job seekers to take a bunch of interviews for positions they won't like/aren't qualified for and won't land -- causing them to have a miserable experience interviewing (and probably resulting in some dropping out of the job market entirely).
[0] The exercise was, specifically, write a for loop with an int value that is incremented by one, adds that int value to another int variable, and multiplies the result against the counter, assigning the value to a third variable. Print the result to the console. This was before "Fizz Buzz" became all the rage, I did my best to give a blindingly simple problem to just validate that the person could do something in the language.
[1] https://blog.codinghorror.com/why-cant-programmers-program/
[2] Never mind the details "Must have 4 years of experience in Angular", nonsense -- a topic for another post, but the way we write job requirements is about stupid. I mean, I've seen experience requirements that exceed the length of time the technology was in existence, and I'd rather see a candidate demonstrate an ability to adapt and learn new things than be overly concerned with whether or not they've used a framework that is really similar to a bunch of other frameworks.
Bad employees are constantly in need of placement: people get fired very frequently.
If you're hearing about a particular job or candidate, the odds are they are bad. Most jobs are filled internally or by referral. IIRC, it's only about 20% of jobs that are filled by recruiters and/or cold job applications.
Shady body shops come to mind.
I've witnessed exactly 2 people fired in 7 years.
And anecdotally, the bigger the company the rarer the firings.
Where were you working at that people got fired all the time?
Not true. Firing is a rare event.
If anything, bad employees will end up in a group that is "relaxed" toward productivity and stays there for a while.
As for bad jobs, even bad jobs need someone who is functionally capable of doing them. Sure, you have to be vigilant about filling those positions, but it's rare to encounter a "bad job" in this field that would be fine to have a terrible candidate in.
Regarding jobs being from internal candidates: It's interesting, because I hear that statistic rather frequently and often wonder where it originates or whether or not it has the meaning that is often attributed to it. Again, pulling from my anecdotal experience, though my resume was passed in from an internal employee for the two jobs I was most interested in[1], in one case my resume was handed to a sales person who reached out to me with a question about the company I had just been laid off from -- I had never talked to him or met him before and he lived about 100 miles away and in the other case, my sister handed my resume off to a friend of hers who worked there[2]. This friend and I met for the first time on my second day of work there. I don't even know if the guy is a software developer or not -- and that's the crux of the issue. If the recommendation is coming from someone doing the same/similar work on the same/similar team, that candidate is probably a good fit. If it's from someone doing wildly different work or who has no idea what a good candidate would look like, it's not much better than the blind solicitation from a recruiter. This is further complicated by the fact that many/most companies grant a referral bonus, incentivizing staff to solicit and (possibly) lowering the quality of candidates coming through that channel[3]. I'm really curious about others' thoughts on this, particularly those who've hired for their own companies.
The last issue with the 20% rule is that in companies that are dedicated to filling a diversity standard of some kind, it's hard to get there from internal employees unless you already have a very diverse staff. Reality being non-ideal, you tend to hang out with folks who are similar to you[4].
[0] There are plenty of reasons people get laid off/lose a job that have nothing to do with the competency of the employee. Companies go completely out of business, shedding all jobs or merge with companies where the whole department gets eliminated. Less "great" excuses are role elimination, though, this happened directly to me in January but I was offered a position for every job I interviewed for, so given appropriate context it can be fine as well.
[1] I interviewed at four places, all four offered me a position, two of the places were the highest on my list and I took one of my top choices. The two that were high on my list technically had my resume passed in from someone who already worked there.
[2] That it was my sister certainly helped -- she's been the CFO of a very well known global company and is currently the CEO of a mid-sized company -- her recommendation is trustworthy and people who know her know that she wouldn't embellish anything just because of family relation.
[3] I'm on the fence about this, though, personally. On the one hand, I'm less likely to recommend anything but the absolute best candidate for a role at company I work for if there's not a financial incentive -- it's nearly all downside otherwise. If the candidate works out, it's unlikely anyone is going to remember you sent that resume in. If the candidate is a mess, you get a ding on your record for recommending a fool and a subconscious ding to your own reputation that you associate yourself with fools (unfair as that is, I've seen it). If there's a financial incentive, there's always the "well, we paid him to do that; maybe he really needed the money" option which is bad, but not as bad. I got 11 people hired in at CompUSA when I worked there in my teens because they paid a $250 bonus. I had one really bad (as in, he stole things, and damaged things in fits of rage bad) recommendation that surprised my managers and me (he was a close friend and I had no idea he turned in to lunatic once he was being paid to do something).
[4] Though this does apply on racial/gender boundaries, one of the goals of diversity is to also avoid group think. Taking race out of the picture, you favor people who are like/agree with you and if the goal is to seek people who are hard working/passionate but not cut from the same cloth as the rest of the team, an internal recommendation will miss that more than it will hit it.
If candidates who can't code are actually getting on site, your phone screening is inadequate and vulnerable to "smooth talkers".
Hiring at the company worked like this, provided the resume wasn't passed directly from a member of staff close to that team:
(1) Applicant applied through web/recruiter/e-mail. (2) Resume was reviewed by a generic hiring team who covered every kind of job the company had and none of whom had even the most basic understanding of what a developer's qualifications should be[0]. (3) This same team does a myriad of phone screens to further whittle the pile. (4) Team's manager reviews remainder and often didn't perform a second screen since one was already done[1]. (5) Candidate arrives.
[0] These same people provided many of the "templates" used for the job descriptions, so you can imagine these were similarly bad. This practice resulted in the company, for years, refusing all resumes where the candidate did not complete a 4-year degree program. I pushed for an end to that for dev jobs after two candidates I hired in who had limited/no college experience turned out to be excellent employees.
[1] And in some cases, the manager knew nothing about software development and was put there from some part of the business, so s/he'd be about as effective at a phone screen as the hiring team.
I'm going to have to assess come applicants soon and have been trying to come up with something appropriate, which is why I'm interested.
So you block off 4-8 hours of your life for this job you're really excited about. What is the employer putting up? Generally nothing. They get a sea of responses and they get to pick a few of the best ones. The 100 responses they got cost these candidates 400+ hours of time for no cost to the employer whatsoever.
Take home assignments are fine, but the employer needs to put up something as well. Either pay for the time I spend on it, or have the assignment late in the interviewing process and have employment contingent on passing some known bar ahead of time. That is, I should be the only one doing the assignment and there should be no wishy-washy rejections.
All of the interview processes I've had recently have checked I did the exercise by either asking me to describe what I built with no aid, asking me to walk through the code with it in front of me, or asking me to conceptually expand on what I built on a whiteboard.
Whittle down the pool even further and the ones you think look good on paper, and after an interview (or a series of them), have them come in, on a paid basis.
It'd be a sign of good faith that they're both serious about finding a good candidate, and respectful of your time?
I know some companies do paid stuff like that. If only it were more commonplace.
I did an assignment like that for a company once, I'm still waiting for the money for more than one year...
Note that 5-minute-long task is a good pre-filter on its own.
I wonder if the best thing to do is to simply offer the candidate a choice.
Why is that a bad thing? The prospective candidates aren't entitled to anything, the burden should be on them to prove they have any capabilities for the job.
> What is the employer putting up? Generally nothing.
Totally incorrect, have you ever had to hire somebody? The initial phone screens alone to weed out the 99% of complete incompetents takes many hours/days. Even getting to the list of phone screens, involves many hours of sifting through resumes, where each resume is cleverly designed to hide the fact that the prospective candidate is incompetent.
I echo the grandparent's sentiments, myself, from my own personal experience as well.
>If you want the job, you'll find four hours.
But what happens when everyone starts having these 4 hour screens? Now its a 40 hour process to apply for 10 jobs in a week (and this is before any commitment on the part of the employer). It's not sustainable or fair. And yet, there's no incentive for employers not to do it. So we as job seekers just need to suck it up and be thankful for the opportunity, right?
After interviewing 10 (say two three hours spent on each between contacting, scheduling up etc) it was clear every unemployed with 'can use word' as skill boosted their resume and applied.
I'm sorry for the real developers looking for a job but a company can't dump 20 days work on chasing and weeding liers.
I'm not a fan of homeworks myself but this is what it has come to.
I'd be payng but that puts the company and the prospect in a very difficult position them towards tax laws and towards their current employer non competes, the company toward again tax laws and then there's the absurd of paying for contract work delivered that's 99% of times not up to spec.
Beside, the unwillingness to compromise smells itself and given the choice I'd never get a potential troublemaker in that only thinks time=money and screw everything else.
That little empathy for both sided of the problems sure is a red flag.
Edit: well unless it's a contracting job then it's fine.
we used to give out this http://play.elevatorsaga.com/#challenge=3 and asking for solution to level 3 which can be done in a short time + is the first non trivial challenge.
of course solutions can be find in internet etc, but it's pretty easy to look them up if they smell fishy and it'd all come down crashing at the face-to-face anyway
it's also a very good exercise in real world problem solving and a source of endless technical discussion since there's no single good approach, especially with two elevators to schedule.
Most of the places I've been at were desperate for 'good' people and they couldn't seem to find them. It's still a seller's market out there.
If this were true people would be poached a lot more frequently than what we can observe now.
Can you, personally, pay me a salary? If you can and I want to work with you, I absolutely think it's fair to ask four hours of my own time to demonstrate I'm worth consideration for it. If not, then why does your opinion matter?
Furthermore, I already have a job, just as most reasonable candidates do - so if you want to convince me that I should spend many hours on that, you'd better be sure that your offer has a really attractive first impression and seems likely to be better than the job I currently have, otherwise you'll be self-selecting your candidates only from the (generally less skilled) sub-population of workers for whom "If you want the job, you'll find four hours" is true because they actually are desperate for jobs, unlike most skilled people in tech industry.
I have never seen a job posting I was sufficiently interested in to put in 4 hours with no investment from the company.
Regardless, if you have a good job already, why apply to any position you're not particularly interested in and that you wouldn't spend four hours trying to get?
Why would I spend any of my already very limited time if they didn't make sure the process is streamlined?
Also, I don't think it's uncommon to like what a company does and being interested without yet knowing if you're interested enough to put the time aside - most company sites and job descriptions are much too generic and vague, so I'd want to talk to some employees first (remember, interviews are two way conversations). So I might not know if I'm interested enough at the time of application, just that I might be.
[1] for my current job, the interview process was essentially them trying to persuade me to join them, and my previous job I was actively looking for a job and someone I know at the company said I should apply. My "application" consisted of telling him "ok set up an interview"
The standard process where the first two steps are some ordering of a 1-hour interview with someone technical and a chat with a recruiter is perfect for this. With minimal effort (maybe 90 minutes) you find out whether they're interested in you at all, and get to ask both a technical person and a recruiter about the job.
The big time investment typically comes later, and I can easily bail if I decide it isn't going to be a compelling offer. I've done this quite a bit.
I'm glad the homework process worked for you, but my point that it doesn't seem like a good idea for the company stands. If the first thing you do is filter based on how badly applicants need a job, you're losing a significant portion of the best people right off the bat.
So now I'm doing a day and a half of unpaid work for them, taken out of evenings and time with the family.
Not hypothetical. I just described my last job hunt, 16 months ago.
The place I really wanted to go closed the req on me, and then opened it back up the day after I started at another place. My third choice (stars didn't align on the other 2 interviews, in part because of stuff like this).
It goes both ways.
EDIT: I want to provide an anecdote around this. If your salary is $150k, it may cost you roughly $400 in gross pay to interview somewhere. Sure, you could take paid vacation, but vacation time is paid out in cash when you leave so it still costs you. Companies with unlimited vacation are a different story.
I had a similar experience a month ago. It was a similar "4 hour project" but it was open ended with "make it as complicated or as simple as you want". Being a single table backed CRUD app, you could only put in so many capabilities but it got me excited thinking about ways I could surprise the staff. In the end, I made it self-hosted (in a language that self-hosted is less common), included an installer, and made it support multiple connections from different devices with updates that pushed to the connected clients. The project took me about a day and a half (with two of those hours spent yelling at WIX), but the core parts took about two hours. I was genuinely surprised at how much fun I had putting it together. They also asked me to estimate my time spent -- since I had contract work I was doing at the time and couldn't dedicate a long stretch all at once to it -- I broke it down into its individual tasks and documented the whole thing. They had no problem with the fact that I delivered it a week after the interview.
I also second your "I can spare a half a day for a prospect" attitude. There's a flip-side benefit to that, as well. A candidate who feels over burdened with a simple assignment (provided it's tested as you mentioned and truly is simple) is probably not one I'd be interested in hiring, anyway. I'm not saying it's "one size fits all" -- I've had candidates walk in the door that I knew in advance were really good fits who were hired after a 30-minute screen. That, though, is a unicorn sighting.
If 10 good assignments meant at least 3 offers, it'd be a different story, right?
In particular, it seems to me that take-home assignments are strictly better for candidate scheduling etc than in-person interviews, if time spent can be held equal. Maybe time spent is just not equal very often?
So you block off 4-8 hours of your life for this job you're really excited about. What is the employer putting up? Generally nothing. They get a sea of responses and they get to pick a few of the best ones. The 100 responses they got cost these candidates 400 hours of time for no cost to the employer whatsoever.
Take home assignments are fine, but the employer needs to put up something as well. Either pay for the time I spend on it, or have the assignment late in the interviewing process and have employment contingent on passing some known bar ahead of time. That is to say, I should be the only one doing the assignment and there should be no wishy-washy rejections.
Unless you've been in the shoes of a hiring manager, I'm unsure as to where this idea is coming from?
Having been a hiring manager, it is a horrible experience. Unless it's extremely naive, homework sets generally take time to come up with and formalize, and certainly take time to review the results for.
I suppose I can understand conflating the motivations of an interviewer with their employer, but it's not generally true to say that interviewers are just trying to exploit those they interview.
It's a one-off effort to come up with an assignment. So the cost per application can be extremely low. As far as evaluating them, the bad ones can usually be tossed out in minutes or less. Comparatively, the cost per applicant is basically nothing.
How do I know you wrote it and didn't merely paste your name over some project I've never heard of?
1) How many forks and stars do you have? 2) How often have you opensourced? 3) Have you blogged about them or created a documentation? 4) Why did they build each of those repos? 5) Commit logs... etc..
Very easy to develop such a framework.
One or two repos can be copied, but there is ample signature on a genuine open source contributors.
There's nothing wrong with that, it's a competitive market, those who get hired are those willing to compete, not those who complain about unfairness about being asked to demonstrate their skills. Employers spend tons of time filtering candidates, even reviewing the submitted work takes time and effort, saying it's no cost to the employer whatsoever is not only untrue, it's naive.
What?! Up until that point, typically, the employer had crafted the job posting, posted in on various channels, reviewed the responses, talked to promising candidates, and then give you the assignment. And a good assignment needs to be carefully crafted, a process that usually takes an order of magnitude time more than what it takes to prepare an answer.
I think your last paragraph is very close, but I think the thing that matters is "no wishy-washy rejections". Not being the only person taking the test or how much time the company spends seem like ways to make that happen to me.
Maybe I'm just completely wrong here, but I think most of the objections to homework type tests are being caused by processes where the homework test was in addition to the in-person whiteboard interview, not instead of it.
How would you feel about a process that was: some preliminary screening -> nominal 4 hour homework thing -> in person meeting to confirm that you're not a complete asshole and negotiate salary?
It is a sellers market. I get recruited every single day. Any company that disrespects me and wastes my time isn't going to be very successful at convincing me to work there.
Developers are the ones with the power in this market, and I have no problem throwing this power around.
Maybe just open separate channels for people who have an open source portfolio. I have about 30 repos in my OSS, but still have to take these assignments. Its such an utter waste of my time.
That is a huge red flag, and means that they will disrespect me at work as well.
So that's an absolute floor of an engineer-hour per candidate: Anything less than 30 minutes to read through it and take notes is useless, and figure another 30 minutes minimum for a call.
If they're not going over the responses in detail and then talking through them with the candidates, then they're just wasting their own time because they're not actually judging anything.
And yet companies have an incredibly hard time hiring talent, even with so many other fish out there. Odd, that.
People need to start understanding their value as an employee. You create more value than you're worth; that's why companies need workers. You don't need to bend over for a potential employer. Plenty of other fish in the sea of employers, too.
And I am honestly just an average SF engineer.
The current engineering market let's me choose whatever company I want.
So, I am not going to waste my valuable time with companies that disrespect me, when there are a hundred more that I could be talking to instead.
The way that a business relationship works, is that the person with the power gets to make the decisions. And in the market that I am in, that person is me.
And I have no problem demonstrating my skills. I just demand that there is another person from the company, on the other end of the phone line or other end of the table, investing the same amount of time into the relationship as me.
If a company thinks that it is unreasonable for me to demand an equal time investment from them, well sucks to be that company, because as we have already established, they aren't the ones with the power, so their opinion doesn't really matter. They can go find someone else to interview.
Oh most happily will, even talking to you is a pain, I wouldn't work with you if you paid me. You're arrogant and entitled, you'd make an awful employee.
If the "homework assignments" are just a scheme to get work done for free, or a competent engineer's time is not set aside to review them, then that's a different story. But as a fair test of programming ability, administered in good faith, I don't see the problem, and such a test is important.
Ask to see the company's employment agreements or contracts in writing first.
I was once given a 10 hour assignment, which indeed took that long to complete. Afterwards, the company expressed interest in bringing me on, and sent me a work agreement to sign, which included many unsavory terms such as being paid for completed work 90 days out.
Since their terms were shit, I didn't take the job, and lost 10+ hours of my life.
Overall level was pretty low that after reviewing about 20 homework assignments, and I had wasted a lot of time on post-mortem interviews where I just had to politely decline with a short code review (after all, they spent all this time, so they deserve something in exchange). After all, I started asking FizzBuzz (yes, not changing the name or anything) on the first phone interview to save time. And yes, turns out, it's not a task that any candidate can do.
So, yes - homework assignments are great, but make sure that a candidate at least knows how to write a simple loop (and yes, I overlook off-by-one errors).
The kicker though, like someone else below mentioned, is that we would receive submissions that didn't even compile.
What I liked about the HW problem is that even something simple gives you and the candidate something to talk about. Why did you do this or that. Tabs or spaces ;) etc...
Remember, the interview goes both ways. You can look dumb for your choice of questions just as much as the interviewee can look dumb for their choice of answers.
- The homework assignment wasn't open and public: it was an option after doing a brief (30 minute) phone interview.
- The other option was basically coming in for onsite technical interviews.
1. The "assignment" took me about 4 hours, which they expected and told me off the bat. 2. It involved mostly relevant code with 1-2 wrinkles that were not odd just not everyday sorts of things.
For the proceeding interview, I was able to sit with 3-4 engineers for 1-2 hours really just diving into details about my experience and their processes, etc.
So if I could offer advice it would be to find a "homework problem" your existing crew could build independently in 4 hours or so. As in you've actually had more than 1 team member attempt the request, and they did it in about 4 hours each.
This was actually an interview process I didn't mind. I have 15-ish years of experience doing development. I can spare half a day for a prospect no problem. It was actually kind of fun.
4 hours seems to be a good "sweet spot". You can ask something sufficiently technical and oriented to the daily work without being a bore. And when the interview comes around, you can make sure it's not some ego-bound contest, more of a conversation.