However, this second route also comes with a number of issues. The most annoying of which in my experience is the amount of time investment each interview requires from the candidate.
At least in the traditional technical interview, interviewers and candidates tend to be roughly equally invested in the interview process in terms of time spent, so interviewers have to value candidates' time because failure to do so will end up wasting their own time as well. In take-home projects, this balance in time investment completely breaks down, and thus there's very little left to discourage interviewers from issuing ridiculously time-consuming projects.
I've done a few of these in the past, mostly to get some practical experience with a new framework & ecosystem I'm not too familiar with yet, but I think in the future I'll likely just politely decline any projects that looks unreasonably time-consuming, or at least try asking for certain adjustments to the spec first.
[Edit: corrected typo]
For example I had one interview where I struggled to complete a WPF test project simply because it was coded in a way i've never done before using controls I've never used and the test was estimated to take 10 min. Meanwhile in my bag on my laptop I had a complex WPF solution i've been working on in my free time for months as a possible product to market and sell which more than showed my competence in the platform. People in my interview didn't even want to see it on pretext that they have no way of knowing it's my code.
Waste of time also, especially if you factor in a 2 hour commute each way.
It's generally inappropriate for someone to ask you to do more than a day of work for a take-home hiring exercise.
They might have a different point of view as to the quality of your work. Or they may have taken other factors in consideration (your portfolio, communication style, etc).
But not contacting you with any kind of a status afterward is definitely shabby, on their part.
And asking you to do a second assignment without taking a moment to evaluate your first (and apparently without bothering to tell you in advance that not one, but two or more "quick" assignments would need to be done), all the more so.
There's also no disincentive for interviewees to spend an unreasonable amount of time on the project. So the test is biased against employed people and/or people with kids.
This can be easily countered though. Send out the assignment at a predetermined, convenient time and require it be returned an hour or two later.
Except that these places very frequently tend to either (1) misstate the problem in some major or minor way, or (2) wildly underestimate the time required to produce a professional quality, bug-free, bulletproof-tested solution. Which can be easily countered by having one of their own team members sit down and take the test first. But of course, none of these places ever do that.
(Well, not "none." I'm being hyperbolic. I just mean in that in general, they probably estimate the round-trip time on these programming guizzes the way they do micro-projects on their own jobs -- as in, "Oh, I can do that in an hour" -- but in real life, it often takes 2x-3x longer).
Pull down this data from Instagram API and create a tagcloud. Should only take a couple of hours.
Except working out how to register and authenticate Instagram's API took me over two hours, than after faffing about with it I realized I only had some sandboxed version that returned metadata and not the actual data I was looking for.
The task would probably only take a couple of hours if the whole environment was set up, but the set up was the problem.
Hackerrank is even worse. They have strange ways of wording the questions. I have to Google around to work out how to use their input and output (I work with databases all day long, not reading and writing to STDIN/ STDOUT). No step through debugger (which makes a lot of sense for the algorithmic type questions they ask). Cut and paste only works in some browsers.
No one expects you to set up a database and connect to it in that sort of timeframe - yet I would probably manage it better as I do it regularly.
That was my rough estimate for the prevalence of this craziness, to.
...a math PhD told me "hey, I got one more ya..." and proceeded to give me a mis-articulated mathematical search problem which -- going by his own statement of the problem, ended up having as its "solution" -- an empty class.
At my employer we send out homework exercises, and I personally did the backend developer exercise before we sent it to anyone. I did this specifically to test how long it took. (For the frontend exercise, we didn't have anyone skilled enough on staff to do it, which is why we were hiring a frontend dev).
(1) It may seem counterintuitive to some, but my own general policy is, when it comes to little stuff ("how long did it take you to do X"), you just have to trust people, to a certain extent. Sure, some people may blatantly or grossly lie. But most likely these people will reveal their slipperiness in other ways, very very quickly.
Meanwhile -- and I think people are quick to overlook this -- playing the "policeman" role in every transaction with the candidate brings substantial negatives. By definition, it's adversarial. And generally there are (nearly always) non-adversarial ways to get the same information about the candidate ("Are they basically honest?") you're looking for. They take creativity (and an ability to read emotions and pick up on other signals), but they're there.
(2) Much bigger -- really, there's no need to sweat about the time to completion at all. Just look at the quality of the code.
It all just comes down to the fact that everything is interconnected: Good people generally turn out good stuff in reasonable amounts of time. When you're looking at good code, there is, I find, an intrinsic aura of ease and comfort which shines through it -- such that you just can't imagine it took them very long to produce it. Everything just flows -- just like it does when you talk to them.
Mediocre (and dishonest) people, on the other hand... never produce good stuff in virtually any amount of time. Sure, they can take the whole weekend to polish off their code... but it will still look bad, or at best, "Meh".
There might be some false positives (or outright frauds) by this approach), but I suspect very few. And those that do slip through, are easy to spot by other means (such as asking them to talk about their solution, for even a couple of seconds).
As to how likely it is that people will lie, given that we've hired several people who did this homework and they've proven to be as competent as we believed, there's at least some anecdotal evidence that some people have not lied.
You have to ask someone with no skin in the game.
Beyond that, I think "a few hours" is a bit too much to ask for, especially since you are presumably following up with an in person interview centered around the assignment. That's a big time commitment, and the commitment on the assignment especially is very asymmetric.
I'm not a big fan of assignments for this reason. They are just so asymmetric. I've been given interview homework that was supposed to take "about an hour" and it was more like 4 to 5 in reality. It felt like an unreasonable time request. I did the assignment, and did in fact tell them it took considerably longer than their resonates. I was invited for an on-site after all that, but declined for other reasons.
It's how the universe works, basically.
Isn't asking people to fly out for 6-8 hours of interviews (which effectively takes 3 days minimum out of your life) a much greater burden than spending 2-3 hours on some coding which you can do at a time of your choosing? (Followed by 2-3 hours of video chat interviews spread across multiple days, no flying required).
Also, just to clarify, the homework problem is _not_ something related to our business. The solutions are of no business value to the company.
I am not saying that you are wrong, but applying to Google does sound like it will have many benefits above your average company.
As to how much we pay ... Our pay is very good, and all but one of the positions for which we've had the homework requirement have been telecommuting as well. It's actually a pretty desirable place to work if you like good pay, telecommuting, working at a small company, a low pressure environment (we've been profitable for many years), and various other perks (training budget, flexible hours, blah blah blah).
There were no other limitations put.. "bonus points for stream based input" "bonus points for test cases" ... only a couple people actually delivered a "working" solution (one that ran and outputted anything), none of which had the correct output (one was close enough), and none had any tests.
It was truly something that should take a "skilled" developer a couple hours. And not something that should be entirely alien. To say the least, I was really disappointed with the results. I did the project myself in about 2 hrs, with test cases, 100% code coverage. (before I even gave it out, it was as trivial a challenge as I could come up with for a real world problem).
Why should I have to pay the couple dozen people for their 3 hours, when none delivered a correct solution?
In the end, the person with the closest to correct solution, was the one with the least experience... that person got the job.
And I had someone on my previous team make a point of bringing up this fine point at a meeting, also ;)
This honestly sounds like a great idea to me, except maybe with a slightly longer time allowance to remove some of the pressure. Definitely hoping more interviewers will start to adopt this method for take-home interviews.
But this method also hinges on the interviewer's ability to design projects that can be completed in a reasonable amount of time and still give good insight into a candidate's skills. I think it's safe to say this will be a difficult task for most interviewers.
It doesn't need to be a time trial. If you're impressed with the code and hire the candidate, worst case is you get someone who takes a little more time but writes great code.
Having a time limit of a day or two is fine if the actual project should reasonably take a couple of hours, but if the project actually takes more than a whole work day to complete, then that's a different story (in my humble opinion, anything that would take more than a couple of hours is an unreasonable demand on the candidates time unless you offer some kind of compensation).
I don't see any problem with this, as an interviewer. The take-home is supposed to be an example of the work the candidate does, they should take however long to do it. I want to see the best-case scenario of the code they write (given the problem at hand, etc, of course). The entire idea is removing the time pressure.
The issue for the interviewer is missing out on good potential hires because your selection process is biased against people with little free time.
I have a family and I'm doing part-time study in the evenings. If I'm looking for a new job, then I can probably find time for 1 exercise a fortnight. If one company tells me they have a "4 hour assignment" and the other a "1 hour", then I'm far more likely to do the 1 hour exercise and pass on the long one.
And if I do the 1 hour test, I'd expect to be assessed accordingly. If you're comparing one person's output after 1 hour with another who actually spent 5 hours on it, then you will be more inclined to hire the person that spent longer on the project, even though that's not really going to corelate with on the job performance.
Don't get me wrong everyone takes whatever time she/he needs and comparing results and time spent is hard unless you only look at "does it work" which in many cases does not do the work justice (and may not even be the most important metric in the long run). For that reason take-home assignments may be better than the almost comical interview often described on HN but they have their flaws that make them far from ideal as well!
I think it is hard to judge the true performance of a potential employee in a company team without actually having the candidate be part of the team (and even then it'll take a good amount time before someone settles in). Some folk may not be the best programmers but are good catalysts in a team, smoothing relations between other colleagues and increasing team output overall. Or they might have a habit of happily taking up tasks that are wildly unpopular and thereby, even if they are not the most performant, solving problems colleagues or perhaps a faster candidate wouldn't have solved. I could go on about this but I think it's clear what I mean.
There is a lot more to a role as programmer than just programming and that is often completely neglected in these discussions.
Some companies are trying to answer this issue by signing a potential employee on for a 2 week "trial," where they hopefully get paid. The trouble there is figuring out how long a trial really needs to be to get a good idea of how that person works and fits in with the team -- too little and it's still a crap shoot, too much and you've already essentially hired them.
In the meantime, test trial runs only work for developers currently out of a job; how do I skip my current gig for 2 weeks to go sit at a potential new employer's office? I certainly have no safety in quitting to go do it since I may get dropped after that two weeks, and there's only so much vacation you can take before you run out of personal time.
Maybe the least worst answer is to have people who have been through the hoops before, and can empathize with candidates, running the hiring processes.
1. Show-off quizzes are of limited relevance.
2. Take home work is a huge stressor for people with limited amounts of outside time.
3. Open-source contributions are a) too restrictive as a filter, and b) favor people with lots of time, just like #2.
4. Personal connections narrow the pool that you have, promote nepotism, and tend to exclude people who are already at a disadvantage.
5. Looking at previous jobs screws over lots of junior people and just means you're depending on the last interviewer's shitty decision.
I read it and told them (a) I had no interest in a firm that behaved that way, and (b) I had no interest in a firm that didn't understand what configuration management tools were for.
When I went through the their take-home interview process, there were 4 projects to choose from, with only one having anything remotely to do with my area of expertise (it was a multiplayer game, and I was looking to work as a web front-end/full-stack developer). For all their talk on how you should select a practical project to talk about because it correlates better to ability to get real work done, it's really rather ironic just how utterly academic and unpractical the projects they offered were (that game was literally the most practical one on the list).
Now they tell you that you're expected to spend at most 3 hours on the project. This might be true for some of the other more academic projects on the list if you had any expertise in the respective areas, but it definitely wasn't true for the multiplayer game project I chose, which had a non-trivial front-end and back-end component, for which testing alone could easily take 3 hours (Or maybe I'm just bad, which is definitely possible, but I've asked many of my more experienced peers how long they think a project like this might take for them, and the lowest estimate I got was a whole work day of 8 hours).
Now they also tell you that it's OK if you don't finish the project. There was also a second part to the interview where they may ask for extensions to the original project, where they'd give us more time to work on it, so I assumed if we don't finish, they'd ask for us to finish the original spec along with some extensions for the second interview. Since I had already finished the front-end component for the game, and had worked on the project for well over the expected 3 hours, I decided to call it a day and work on preparing for some of the other interviews I had that week.
However, that it was OK to not finish the project for the first interview might have been an outright lie. I won't ever know for sure because I was rejected after the first interview for not finishing the back-end as well as being unable to resolve some performance issues the interviewer pointed out (which at no point in the interview did he even ask for me to fix, so I assumed fixing that performance issue would also be a part of the extension). Incidentally, I went back to the project a while after the interview and resolved the performance issue in 5 minutes flat.
The whole process just left a rather bitter taste in my mouth, which was all the more disappointing because I went in with great hopes after reading all their great blog posts on HN. For anyone else considering Triplebyte, I'd highly recommend going with their traditional interview route until someone at Triplebyte can confirm the process has changed for the better. At least with that route, if you get rejected, you'd have wasted less time in the process.
Yeah, they always say that -- but it's never really true.
They should just be honest and say "If you don't finish the project in time -- then don't feel bad, but perhaps the test isn't right for you, at this time. Feel free to apply again in 6 months."
I don't understand this. No company is so special that I would be throwing away hundreds of hours in pointless meaningless work every few months just to join them! Unless I'm in need of a job again.
And whats the big deal even if I join them after six months, I'd be working to maintain some code, fix bugs and may be occasionally do a big important project.
Its not like they are sending Neil Armstrong to the moon all over again that I would like to be a part of this history.
After being invited twice for Google interviews, I mean really invited by their HR, not me applying for them. On both occasions I failed the process with their stupid questions.
I started replying to their HR, if I am so good to be invited but on their eyes unable to devise a inode search algorithm for unlimited hard disk sizes with a specific set of hardware and search time constraints, over the phone interview, then why couldn't they just please stop inviting me!?
That was the last time I heard from them and I don't care a bit about it.
Curious -- was this problem reasonably related to the kind of work you'd be doing in the role you were applying for?
Or did they just want to find out if, you know.... you had that "spark"?
But thanks. Another data point added to what others have been saying about their hiring process.
I do not know who you are (and would of course not post details here), but from what you say, it sounds like we did not think that you were on track to finish, and has some concerns about the design that you selected (we track whether a number of milestones in the game have been reached). It is totally possible that we were wrong. We much prefer to see a working front-end / back-end combination that has missing features than we do just a front-end or just a back-end.
Again, I apologize for your bad experience. I hope we can make the take-home interview better in the future with some tweaks.
None of them actually tried solving it within the constrains that a candidate is put through.
Larger projects take up more time and introduce a larger possibility that some matters of opinion or taste will impact the candidate's performance. I don't believe differences in taste are important as long as the candidate demonstrates that he is able to comply with a defined style guide.
The actual test is going to depend on the position at hand, but one of my favorites is a very simple program that asks the candidate to use GitHub's API to display the list of public repositories under a user-inputted username. That's it. I tell them they can use any language they want.
This simple test gives all the information we need about the basics:
a) the candidate is able to go online, provision himself an API key, find docs, and reference those docs to see an external vendor's API format
b) the candidate is able to use that information to craft a program that successfully interacts with the vendor's endpoint
c) the candidate is able to present the information in a concise, desirable manner.
d) the candidate is able to do all of this with a simple 1 paragraph description of the project.
The choices the candidate makes in the process of completing this simple task tell you a lot about his process, style, habits, and preferences, even though the project is very minimal in its actual requirements.
I've had people give me web apps, command-line apps, and GUI apps that accomplish this same goal. Many candidates would go above the requested specifications and many candidates would reply the same day they were given the test, which to me was an excellent signal that they felt their time was respected and that we were doing a good job of engaging them and making them interested in working for us.
As I stated, this test is not appropriate for all positions, but I think most tests should be modeled after those principles. Give the candidate room to express himself and demonstrate relevant practical knowledge.
I hope you'll consider a minimalist project like this over something like "design a multiplayer game ... in 3 hours".
Have you considered requiring a thought process journal as well?
In terms of project choices, to me, it seems like you guys erred on the side of choosing projects developers would find interesting and challenging technically over projects that are practical and accurately represent the kinds of work most developers will actually be hired to do. I'm not going to post the details here for obvious reasons, but out of the four projects offered, only the multiplayer game was even remotely relevant to front-end/back-end or even application development in general, which are areas that probably account for the vast majority of development work available from startups. I realize there needs to be a balance to be stuck here, but in my humble opinion, as a recruiting firm, you should be erring on other, more the practical side in terms of project choice. I applied to Triplebyte to find a job, not to fulfill my intellectual curiosity (I can do that better on my own time without needing someone to assign projects to me).
In terms of project scope, I'm not really qualified to comment on the other choices, because they were way outside my area of expertise, but the multiplayer game definitely didn't feel like a 3 hour project. As suggested in another reply, I really hope you guys can actually give the project a try yourself and see what level of completion can reasonably be expected from three hours of work on something like that. Take whatever times reported by candidates who have successfully completed the project with a grain of salt, because people will have a tendency to understate the level of effort they spent to make themselves look more efficient (no matter how much you tell them you don't care). Here's another possible idea for making projects that take reasonable amounts of time to complete: just take your traditional interview questions and slightly extend them a bit with extra features, and simply expect better polish, architecture, test-coverage, and overall code quality, etc during the code review.
Anyways, my experience with Triplebyte's take-home interviews definitely didn't leave a great impression, but I still recommend you guys to my friends and colleagues because I do want to support what you guys are trying to do. Hopefully you can take some time to revisit some of the issues people have mentioned and make the necessary improvements. I'm happily employed now, but I'd love to give Triplebyte another try the next time I'm looking for work. =)
I'm now in a position where I'm interviewing and helping shape my organization's hiring practices. We've debated all the different approaches, some people like projects, some like algorithms, and some don't want to do either to get the job. At the end of the day, I really just want data on a candidate's ability so that I can say Yes.
There probably lies the disconnect. For me, interview projects should assess how well I could perform in the position I'm applying to. And thus, if nothing else, interview projects should be relevant and practical.
It seems to me that Triplebyte's project choices were made based on how interesting developers might find them, and sheer technical challenge. Some might appreciate this, but personally, I'd rather learn new skills and challenge myself on my own terms.
Not really true unless you go to your interviews completely unprepared :-)
All in all, it's at least an hour of prep time needed, and that's only counting my time, not the time of the other interviewers, managers and recruiters.
I had a friend of mine who told me her company doesn't use online programming tests (like hacker rank) because she doesn't think legit programmers would bother with positions that required them.
Having taken a couple of these on-line automated tests, I don't think I'd take them again. Problems which I'm sure I had written correctly, dealing for edge cases and testing in browser; I submit to get like a 6%. Programming simple things in front of people in an interview I do fine.
An hour, I'm fine with. It's less than what I'd schedule for an interview, and far less than I'd schedule for an in person interview (which might include a flight out). On the other side of it, though, I'd be concerned about cheating. It wouldn't be too hard to hire someone to take the test for me, I'd imagine.
Then in the rare case they question it, which usually goes something like, "Can you do it in Java? We don't use whatever language it is you're using there." I respond with, "Oh, will I be spending a lot of time at this job coming up on-the-fly with somebody's PhD thesis from the 60's?" To which the answer is always, "No." Which finally ends the conversation with me saying, "Well then you asked me to solve an irrelevant problem, so I'm happily providing you an irrelevant answer."
I have a fair bit of contempt for the pitiful state of technical hiring and assessment process out there. I think that comes mostly from my opinion that hiring is probably the most important thing any company or manager does, and that if any one of these companies/interviewers put even 10% the amount of effort into learning about educational/occupational assessment, psychology, and neurology that they do into obsessing over the dubious idiosyncrasies of the latest framework-rehash-of-the-month, then these terrible interview practices wouldn't endure very long, and we'd all be spared the indignity of the farce on both sides.
So how many of those companies made an offer?
ressource AClass::CopyACriticalRessource()
{
{
std::lock l;
} // Optimization for the lock.
return ressource_;
}
The best technical interview being a homework with enough time for a normal human being, requiring tricky algorithms to solve it in a fancy way, a good architecture and great computer-science knowledge in general. Afterwards, a code-review/interview of 1 hour with debriefing on all the choices.
On the flip side, my experience with take home projects is that they are much more of a hazing/pressure cooker ritual - at the least, the time pressure of a Google/FB/etc. interview lasts only a half hour. Most will expect candidates to spend a huge amount of time, which many won't have if they're actively job searching, or are busy people in general - I prefer to pour that extra time into open source work, since at the least it benefits others. One company assigned a project to me once that suspiciously seemed like implementing the company's whole business model.
It doesn't take long to figure out if someone has the right skills oftentimes if you ask the right questions - candidates shouldn't be penalized for companies' ineptitude in assessing interviewees.
As someone who pushed his company to adopt a "take home" assignment for our interview process, it should be perfectly reasonable to reply with "here's my commit history on a project relevant to what you're hiring for."
I much prefer giving (and taking) take home assignments because it lets the interviewer see _what someone will actually produce on the job_. If you can do that without jumping through our specific hoops, great. If not, that's why we have the assignment.
It also at least somewhat ameliorates the pressure cooker of an interview, which many people cope with poorly. It can be hard enough to communicate clearly when all eyes are on you, let alone get your thoughts together and solve a logic problem. If that's actually analogous to your work environment, well...
Consider this: We need to find the most shared URLs on Facebook in last 24 hours.
I can perfectly see why a lousy coder might achieve this objective better than a kick-ass one. A coder with average skills quickly figured out that Buzz-sumo has a public webpage with that content which can very easily be scraped using phantomjs. Job got done.
Another great coder suggested to me I should buy $10K per month Facebook firehose. He is not wrong and that is a good solution too but he failed to see that we are building a POC and not a full featured product.
Needs of companies are complex, many times if you can pay great salary just having a filter for IQ is good enough. But in most cases I think it is far better to look at an individual and judge.
No, but it may be a good question, anyways. A good interviewer probes you about things you might encounter in the job to find the point where you cannot recite things, and looks how you handle that.
Having said that, I don't understand the focus on coding interviews I read on HN. I don't know whether it is cultural difference between the USA and Europe, whether I just haven't noticed how you are supposed to prepare for interviews, or whether I am too smart to need to bother, but when I apply for a job, I look up what the company does, try to figure out what its culture is, but do not prepare for the technical side of things. An interview isn't a one-sided affair of you being quizzed, it is two parties figuring out whether they fit together,
I helped a group within my organization with their hiring process recently, and we had pretty good success with assigning a short "take-home" exercise, vs. trying to haze them with a programming problem over a google hangout interview. A problem focusing on a small part of what that group does, but scoped to be doable with 1-2 hours of work.
Everyone else, this sucks!
"Hey here's a fizz buzz question, if you feel confident answering it now go for it, otherwise we'd be just as happy for you to do it in a take-home fashion?"
Doesn't this have exactly the same problems as homework projects? You can still copy and paste or plagiarize.
The most practical thing is to work with it as it is.
This is actually a trivially easy question that gets to the heart of whether you understand the point of moving averages or not (that you can update a sum by subtracting out the value leaving the window and adding in the value entering the window).
"Programming ability" isn't one dimensional. Whether or not the above question is useful depend entirely on what skills are important in a candidate.
Granted, by this time, we'd been through a couple of other problems, and time was running short, but I still think it was pretty unprofessional of the interviewer to let frustration or any other sort of negative emotion show during the interview. That, more than anything else, contributed to my own frustration and perception of unfairness in the entire interview process.
Or was it a more complicated moving average case (exponential etc) where the algorithm was given and you were asked to determine the complexity?
Figuring out how to efficiently calculate a moving average seems like a good question for basic maths skills to me.
About complexity theory, there is already a lot of discussion about it's relevancy in the comments. My point was meant more along the line that, if you value a basic understanding of complexity theory, the question asked of the GP seems reasonable.
As a developer, I prefer take-home projects. As an interviewer, I prefer a few coding interviews, followed by a take-home project.
This is will allow members of my team (or myself) to get to know the candidate, evaluate the 'wave lengths', and how effective he/she is at finding patterns on the internet/books -- rather than thinking things up.
It also demonstrates to the candidate commitment on our side, and it naturally forces us to find problems of a proper size/effort (as we are spending the effort too).
We would not do it for all applicants, though -- only for once that pass basic screening / competency process (there are no trick questions or exercises there
I have not tried it. But in my next business venture, I plan to actually do
peer programming with the candidate.
It was one of the most enjoyable, and, dare I say it, relaxing interview experiences I've had. Though I didn't get an offer, I wouldn't hesitate to recommend that company to any of my peers who're looking to make a change. Also, like you said, it was a second tier screen; after an initial screen consisting of a more traditional phone interview where I had to write some Javascript.
I imagine there are tons of false negatives, but it's nearly impossible for
a terrible programmer to get through a gauntlet of programming interviews.
But I've found that even once you get past the outright bozos, there are quite a few programmers who can program quick one-off things, but have no sense of design or maintainability. They can deliver functionality, but deliver in a way that piles on technical debt and damages the long term health of the codebase. I think the traditional technical interview format ironically encourages this sort of behavior, by encouraging applicants to focus on narrowly solving the problem at hand, as quickly as possible, both in terms of machine time and programmer time, even if that means the code is an unmaintainable mess in the long run.
Put another way, think back to the last time you had to do any sort of whiteboard coding, as part of an interview. Are you proud of the code that you wrote? Would that code pass code review at your current position? If so, then congratulations. You're a better programmer than I. The code I've written on whiteboards has been pretty uniformly terrible. Sure, it met the correct Big-O complexity requirements, and it was correct, insofar as it produced the correct output, given correct input. But there was no error handling. Variable names were single letters. The functionality wasn't broken up into logical functions because writing additional function headers takes more time, and my handwriting is messy enough when I'm not rushing. All in all, it's code that you'd see in a prototype, or a programming contest entry, not a robust system that's usable by customers.
Lately, I've seen more and more such code being produced by new graduates not only in coding interviews, but also as part of day-to-day programming. There's an incipient attitude of, "Well this code would pass in an interview, so it's production ready." I find it deeply troubling, and my concern is that programming interviews are setting up incentives by which this sort of code becomes, if not normal, then certainly more accepted than it was in the past.
This is why I advocate so heavily for take-home projects. When a candidate submits a take-home project, you can be assured that they had enough time to design and code the assignment in a maintainable way. You can see whether they added unit tests. You can see whether they split the code logically into objects and functions, or whether they smushed everything into a 500-line main(). I accept that take-home assignments aren't as scalable, either from the interviewee side or the interviewer side, but I do worry about the long term effects on norms that programming interviews are having.
EDIT: grammar
> Being a good president has a surprisingly small role in
> being elected president.An interview is for both prospective employer and potential employee to meet and find out about each other. If you as a candidate notice something you don't like during the interview (including the interview itself), that already is a valuable information for you, as you already may assume things based on it (like, what kind of people you may find past the in-process interview as they all more or less were filtered in by it). So you can always stop and say "thank you" and walk away without wasting more time. You'll allow yourself to stay longer only when the interview's quality warrants it.
With 'weighs heavily' I mean that it influences the criteria for which those methods are good for.
The downside of this is that interviews are more unstandardised, which is a trade-off worth considering.
However whiteboarding is also terrible.
I prefer getting a real world / work related scenario problem, and solving it TOGETHER with the interviewer: if I'm stuck, as opposed to a hackerank codility challenge, there is a good chance they will let me know and hint in the right direction, and they will also see how I communicate, how I think, how I respond to hints. An online challenge can be easily done by the candidate sitting next to 2 more senior coder friends who help him / her along the way. Unless it is proctored, you can never know. I had people whispering someone answers during a phone screen, people typing my question in Google and reading me the first result (I google it at the same time).
I think that giving you a hackerank or codility take home problem unless this is an entry level job will simply drive away experienced people.
If someone can pass your take home test easily, you will still need to phone screen them before you fly them over in most cases, and if they are that good, they will probably prefer companies that don't waste their time and jump straight to the real person phone screen.
However, some people are more nervous when someone is watching over their shoulder, so here is what I would do: I would ask the candidate what they prefer:
1. work related a hands on assignment with plenty of time (e.g. should take 30 mins but you give 1 hour) and ability to search online and ability to compile use an IDE just like in a real world scenario
2. Skype screen with a lot of small questions on a topic they really feel they know about (no Googling allowed), and a relatively smaller coding challenge (something you can code in 15 minutes)
3. Work related scenario but with a real person, where there is no one best answer to the question, but more of a balance of tradeoffs and more open ended but 100% involves coding (just like most phone screens, but more work related and not just puzzles)
4. The standard puzzle challenge but alone - you have 30 minutes to solve the problem once you see it (Googling is allowed or the test is proctored to make sure you don't, then you get more time)
5. the classic - a cracking the code interview kind of question, but with a real person, code sharing (for crying out loud not google docs, at least have your candidates use something that offers color coding and easier indentation) - some might choose still this
If you let your candidate chose what is best for them, you already made an amazing impression, and you might have much less false negatives.
Just my opinion
I think this is one of the most inane things to be asked during an interview. personally, I've never found myself in a situation where I truly needed to choose between a vector/map/list/hashmap. Or had to find the O(x^n) and replace it with O(x^2)
Obviously it depends on the application, but many jobs are simply maintenance coding: find bug, fix bug, test fix. Often times it makes absolutely no difference whether you use a list or a vector, or else you'll get the paradoxical "vector-is-always-faster" because of locality of reference.
In my (admittedly limited ) experience, most of the effort is spent simply making it work, not being bogged down because you used a map instead of a hashmap, or didn't know about some esoteric, bleeding-edge probabilistic data structure.
But I think it's more telling if a programmer knows how to profile his program and find the performance bottlenecks, recognize them for what they are, then fix them appropriately than if they can recall a specific optimization for a specific use case on demand.
What you want the person to identify is that they've made your simple iterative checkout process into multiple unbounded tree traversals with no circuit breaker.
Knowing that searching the tree is O(log(n)) isn't very helpful when your problem is an inability to identify that you've made (n) an unnecessarily huge problem space.
It may be something that's so intuitively obvious to you, that you don't even think about it. So you naturally use the hashmap, where someone else might try a list and then start doing a lookup in a loop. Then while that particular instance might not break things, it'll slow things down, so overall the application feels sluggish instead of snappy.
This entirely depends on the kind of product you are working on. When you get to a large scale with any programming project, optimizing computational resources will cut costs, and can often add value to the customer as well.
The other thing that really seals the deal for me as an inferior interview question is that you don't need to have a clue what O(x^n) is to wrap some code in a simple time call, see that the code you think ought to run in microseconds is running in seconds, by visual inspection notice stupid nested loops, and fix it. Self-taught programmers may not be able to say "O of exx to the enn" but that doesn't stop them from fixing it.
So... seriously, what good is the question anyhow?
Then I noticed the test was expressing constraints using time/space complexity, concepts I was completely unaware about for my previous 15+ years in the profession.
So now I am reading about algorithm theory face-palming at the realization I have reinvented the wheel many times during my career instead of just re-using a PhD. researched algorithm.
You seem to be confusing "knows how to say 'oh of enn squared'" with "the loop gets slow when I iterate over a lot of things". One is generally a product of education, the other, merely experience.
The idea that a thing can only be learned in a classroom is perhaps in the top ten most pernicious ideas in the modern world, and probably one of the more surprising ones to show up in that list. You do not need special courses to discover that your code runs slowly, and as I've seen often enough, having had those special courses does not confer immunity against writing slow code.
Now, I am also a believer that formal education has its place, and if you are going to get a formal education in computer science, big-O analysis absolutely must be part of it or you are literally missing out on an entire rather important sub-discipline. But the idea that it's some sort of touchstone between Good and Bad programmers is just ludicrous nonsense. Slow code is slow. There are abundant tools that can be used to figure out why. If you can't work out why your O(n^3) loop is running slowly after a couple of years of practicing the art, you don't need formal education, you need a different job.
Sorry, but this is just wrong.
I can guarantee you that BigO is very often used not "in theory", but in practice, to find real-world performance problems. And not having "situational awareness" of certain commonly occurring complexity profiles can be a significant source for performance headaches and technical debt.
One trivial-seeming, but frequently occurring example: not knowing when to use hash maps.
In fact, many people solve performance problems (including both those where order-of-magnitude performance really is the most important factor, and various other kinds) not by using a profiler but by, you know, understanding the code and thinking about it. Being as profilers, while they can tell you a lot about certain kinds of performance issues, are still generally quite limited in what they can tell you.
Is it the silver bullet people seem to think it is? No.
Of course not, and I've never heard anyone saying that it was, either.
It's like, yeah, you don't "technically" need to know any 2+ syllable words to be a programmer, but you're really not helping yourself by avoiding them.
I'm not saying I'm pro writing-inefficient-code, but if you want to talk big-O during an interview, I"m going to roll my eyes about as much as you asking me who the 19th president was.
Some people really do never advance past a certain point, but a lot of people write someone off as mediocre when they're just still in the process of gaining skill. Also, they only assess ONCE, which is pretty bad if we're trying to establish how good you are forever and always.
I don't think though you need to know that for example fastsortx is n log n in the best case off the top of your head in an interview situation. You should be able to reason your way through why it is faster than some other sort though.
If your job is mostly frontend, yeah, you probably won't need to worry about this problem. But if you're hiring somebody to work on graphics? You better be doing complexity estimates in your sleep.
No -- we like solving problems. Especially those that have a legitimate context (i.e. are explicitly linked to some actual, real, business or social problem). And for which our skills are truly relevant needed (specifically, for which no one has a readily available answer, at the moment).
But made-up "puzzles"? For which the asker already knows the answer, so they sit back and watch us dance?
Not so much.
This method of interviewing has been around ever since, and is going to be around for the foreseeable future. Nobody loves it, including the interviewers, but there just isn't a better way to do it at any sort of scale. Especially when there are much bigger problems to solve when you're running a business.
It's best to take the bull by the horns. I run http://InterviewKickstart.com, which is a bootcamp for preparing for such technical interviews. We do almost exactly what is in the blog post. It works. Spectacularly.
I want to take the time to talk to everyone who is interested in the course. Because the concept is new, people have all sorts of questions. Can't possibly address all of them in writing. I don't have a team of salespeople and haven't spent a penny on advertising.
If you still prefer email, please feel free to send one. It's on the site. It may just take longer to do back and forth.
Thanks for considering!
Then maybe it's not such a good concept.
After all, it's an 8-week intensive course, mostly for CS grads, that grills you hard, and is not cheap. As a consumer hence, I'd highly prefer to talk with someone. Not to mention, all educational institutes have an enrollment process that needs you to talk to a human.
At some point, when it becomes more common and well-accepted, we will condense it, but it feels a little too early to do so.
Pricing is also nuanced based on whether you're an experience engineer or student, whether you're taking the course remotely or on-site. Plus, there are recruiting firms who have access to our pipeline, who return a significant portion of our fees to you directly (we don't take a cut).
And those who are super curious, can always google it :-) In fact, most people who call have already googled for it before calling.
Rest assured, we're a real business, running classes every week. Batch after batch. Those who work hard, are getting their work rewarded.
I don't have the answer, but I just don't see how the questions typically asked in programming interviews give you a good picture of the candidate's actual programming ability. I much prefer "homework" projects, even if they involve me working "for free", because I feel like they ask for actual programming skills rather than the "guess the algorithm" lottery of phone screens and whiteboard coding.