Without something like this we'd just have 100s-1000s of unstructured resumes in different formats to rank against each other. This process is more fair and (IMO) allows someone without a college degree, or from outside the US, to compete on a level playing field.
> Without something like this we'd just have 100s-1000s of unstructured resumes in different formats to rank against each other.
So instead of hundreds or thousands of unstructured resumes in different formats, you have hundreds or thousands of completed programming tests. Even assuming that you automatically filter out submissions that fail testing, given that there are a lot of ways to solve a problem, you'd still be stuck evaluating potentially hundreds of correct submissions.
More importantly, a test doesn't seem to be aligned with your top priority:
> ...but the most important requirement is significant independent programming experience as demonstrated by your GitHub account, personal projects, academic publications, or startup success.
Are you assuming that all of the folks most accomplished in the real world are going to complete your programming test successfully?
Since you ultimately seem to be interested in folks who can build dashboards, I would think you'd be far better off asking candidates to build a dashboard prototype using sample data you provide. There are lot of really smart developers out there with advanced degrees who could solve programming quizzes all day long but couldn't design and implement a useful dashboard application.
I'm guessing the MVP of hackerrank was the web code editor + lint/findbugs/whatever for filtering. A lot of people have spent a lot of time writing open source projects that convert code to metrics, so making a reasonably prioritized list is probably fairly easy.
> There are lot of really smart developers out there with advanced degrees who could solve programming quizzes all day long but couldn't design and implement a useful dashboard application.
Agree!
I think this is a great critique of HackerRank as a business idea -- while they are new, it's just a hurdle that weeds the busier coders out of your hiring pool... if they become established there will be books on cramming for it, and will no longer separate the wheat from the chaff.
(ie https://web.archive.org/web/20130114163457/https://raganwald... meets http://en.wikipedia.org/wiki/Goodhart's_law)
Sorry - programming tests raise far more false negatives than false positives (and this from a person who considers himself a false negative in most tests)
Majority of the tasks IME:
- code reading & refactoring
- figuring out why two libraries aren't playing nice
- bug triangulation
- optimization (not the algorithmic type, but rather the fix-the-auto-generated-SQL-query / frontend asset packing kind)
- repetitive/simple/obvious bits of jQuery
Maybe there's some kind of online test that could exercise those skills, but if you ask me the main performance changers for modern web development are, they're: really knowing the stack (problem anticipation, what to use, how to use the various tools to quickly get search keywords), google fu, and most importantly having done it a couple hundred times (muscle memory).
Yes, binary trees are pretty much coded for you, and you would be a fool to roll your own hashing algorithm for production, but thats still not a reason to keep your head down.
The next decade is going to be fun. Trust me
I guess to reiterate my point: once you've done web development for a couple years, coding most websites is less about solving new problems and more about applying known solutions.
To reiterate how this point relates to the question of coding interviews: in as much as they don't test for lots of known solutions, and the ability to apply known solutions quickly, they aren't filtering for productive coders.
How does your response relate to the points at hand?
Think micro-services supplying REST APIs over HTTP - that's going to be same-y after a few years yes, but behind that is a world of business apps, integration challenges, complex algorithms and calculations - sorry I just see "web development" as "anything that has of will have an http access point" which I mentally count as "pretty much everything from here on in"
Programming tests do generally test for ability to apply known solutions quickly - it's about all they do do. You just have to know the known solutions (oh look calculate primes).
As for finding productive coders - well the amount of false negatives is large, and the false positive rate is probably higher (plenty of people who can waltz through these tests i find fairly poisonous to morale) so I am not a big fan - but if you need to make a first cut this is a good start
- Communicating clearly and calmly with other people
I think you missed my point. It's that the people submitting to a multi-hour exam just to submit a resume are more desperate / value their time less than those that can stand on their experience.
The fact that there a "quite a lot" of submissions only worsens the probability that the multi-hour effort will be wasted.
Just curious, why do you say that the normal process doesn't allow people outside the US to compete on a level playing field? Higher barrier to on-site interviews?
* By well-executed, I mean - relatively language agnostic, lacking in trick questions and meaningless trivia, and being open to multiple good answers. The intention is to not only check basic ability to solve simple problems, but the general approach and programming style of the individual interviewed.
Past experience is often vaguely stated in CVs and looks great on paper, but can be disingenous. It takes a thorough two-way discussion which is customised to each individual to reveal this, however. Some people have excellent and highly visible past achievements, but not everyone does. I wouldn't throw out a CV simply because the individual doesn't have any open source code to show me, nor do they have any awards from companies they have worked for (which is often more about self-marketing than actual ability).
I guess your mileage may vary based on the position that's in question and the companies involved, but I've seen some shocking discrepancies between CVs and actual programming ability. Even worse than the usual "fizz-buzz" statistics that get thrown around.
this kind of test is not valuable. the other tests i've done though - even though incredibly easy i think would be more useful for seeing how i approach problem solving in general...
Regarding CS majors, there's always the possibility of an individual being great at the theory part, but not having much programming experience. This should be rare though; during my own CS degree, we coded so much that I was frequently dreaming about code. If it's provided in the CV, I like to scan through the subjects list of the candidate - the ones who don't actually like programming seem to have an aversion to taking CS courses for elective subjects. As always though, YMMV, and it's a good idea to keep an open mind.
conversely my experience of education is that it trains you to solve education style problems - which are essentially absent from the real world - where being able to invent and implement a slightly broken solution in an hour with 30 seconds notice is much more valuable than being able to solve a problem you know about far in advance and are given loads of information and guidance to help in preparation for it.
on the other hand if i wanted someone to do research then a CS major might be a better choice. :)
In a tight labor market, people willing to do this are more desperate than others. So it's going to filter for desperate developers who passed a 3 hour basic skills test.
Old fashioned prior experience and education is probably still a better first filter. This would probably make more sense specifically for junior hires right out of school or with no experience.