...which is so very frustrating. It's painful to read things like this:
> That was all we asked; we intentionally left it open-ended. Some people tried to impress us with huge, elaborate apps that used different services and engines.
The people hiring viewed this as a trivial, small, meaningless ask. They are proud to have "intentionally left it open-ended."
Some potential candidates did not (through their own example) view this as trivial, they went ahead and poured a ton of time into it and most likely were "replied with well wishes" as the company moved on.
It's irresponsible.
That said, if a candidate spends 12 hours or 10,000 lines coding up tic-tac-toe, the company is not entirely to blame here...
I clearly hear what you're saying, and I understand your stance on it. But I can't agree with it.
The burden is on the folks hiring in my opinion.
This is not a "real life" scenario. It should be a SHORT assesment (for both parties). If you were employed within a company as an engineer and you received a nebulus spec (or no spec at all), the burden is then on you to identify, educate, and communicate.
Not during a pre-interview (for the chance and maybe being blessed enough to get a full interview with this world changing company).
FWIW, many of the FAANGs are famous for asking intentionally underspecified questions, and the candidate preparing properly by figuring out what's in scope and what's not is part of the scoring. But at least in an interview you're timeboxed to 45 or 60 minutes!
Maybe, except when you try to clarify and their answer is that it's open-ended -- and yet they want something super specific in the end. And the super specific things are just one of many tech opinions and not objectively good design.
> FWIW, many of the FAANGs are famous for asking intentionally underspecified questions, and the candidate preparing properly by figuring out what's in scope and what's not is part of the scoring.
Why is this a good thing though? Why is solving riddles in a professional interview somehow ok?
Hmmmmm... I'd say that "my point of view" differs from yours, and does not fall in line with common group think. Cognitive biases are typically pushed back on, even when they are in your favor.
> since it's the candidate who's the prospective engineer, wouldn't that imply that it's their job to clarify the nebulous spec
Gosh, no. You have it backwards my friend. It's a real opportunity cost on your part to think like that. It's a tactic for some to offload the work onto others so they don't have to spend the time doing it.
> FWIW, many of the FAANGs are famous for asking intentionally underspecified questions
99% (or more) of the startups/companies out there are not FAANG's. They are not the litmus test on how to effectively interview. They have candidates FIGHTING each other for a position at their company. Not the other way around.
Even moreso if you only have nights/weekends to work on it.
That’s it. No further instructions.
How happy would you be with the instructions from that teacher? What advice would you give to your child?
While the company isn’t entirely to blame for how much time is spent, they are being irresponsible to a degree.
I personally hate it, where I work we do this to people and you can tell just how long they spent on these things, and god knows how many they have had to knock out as part of a job search
I tried to end the practice but got a lot of resistance
Leaving people to guess at their own evaluation criteria is disrespectful, and implicitly biases in favor of people who have lots of free time/energy.
If you are expecting a candidate to have a certain skill-set, having a simple timed coding test as the first round is the easiest way to filter out a lot of unqualified candidates. Stripe does this quite well for example.
When I hire for my company, I do a live coding round with the candidate after an initial phone screen. Since I get far fewer applicants, this works out for me. The objective with the coding round is to understand more than their ability to write code in a particular language. It's a good way to learn how they communicate, how they approach a new problem, how they seek help, etc
The problem is 99% (or more) of the rand-o startups/companies that do this are not Stripe. They are not companies that people are fighting to become employed at.
> this works out for me > their ability > how they communicate > how they approach > how they seek help
What's ends up happening is you miss out on a lot of truly talented folks by putting the burden of proof 100% on them.
> is the easiest way to filter out a lot of unqualified candidates
It may be the easiest way for YOU. It's an important distinction as you (and this is not a personal attack, sorry if it sounds like it is) may not have the hiring skillset. It is a skill. Just like managing people, supporting software, being a CEO, etc...
I've been asked to write a full-stack app from scratch, and I spent like half of the time debugging the webpack config, which could've easily been solved by having a template prepared in advance if I knew what I was doing.
If anyone's reading these, please have the coders work off of an already existing template that you give them.
As to the coding part, I have interviewed some people who seemed very together, but they could not write a simple program. I do mean simple, not some brain-cracking gauntlet of a test. Something you let them work on, in front of you, for maybe 10-15 minutes max.
Make a loop that sums up numbers in an array.
Write an algorithm that sorts a list of three numbers. I watched students spend an hour making a bunch of if statements to try and cover all cases... ultimately getting it wrong.
Compute the accuracy by computing the number of matches between two lists and dividing between the length.
Split a list into two lists.
Generally the failure mode is to find something vaguely near the task, but overly-sophisticated from google. Then wrestle with trying to hack it into working by frantically scouring stackoverflow etc for their next code adjustment.
A lot of frontend devs don't even work with the DOM directly after having spent years with React. Could they do it? probably. Are they prepared for DOM-related manipulation questions on a whiteboard? nope.
And suppose there are a lot of cheaters out there who don't know the things on their resume, even including earning a degree without ever having to write a working program on their own. How would you screen them out?
At my previous role we were surprised just how many people looked good on paper for a development role but barely even attempted the first question on a 3 question HackerRank test.
Oh and that previous paragraph? That's how I explain our coding test, I'm very clear what our standards are.
>How can you possibly evaluate a test without predefined criteria for evaluating that makes the result good and what makes it bad?
If it's even a slightly reasonable solution it's a pass. It doesn't need to be "good" it just needs to not be terrible.
>Another down side to coding tests is that if the assessor knows less than you about some area of coding and sees something they don't recognise or understand, then they might well negatively view what in fact is a much better way of doing things.
A correct solution is acheivable in between 50-100 lines of code. If you do something I don't understand I can look it up. In fact, people have done things that I don't understand and looked up.
>And, much worse than anything else, you have a very tangible chance of spending 12 hours doing it and hearing nothing back at all.
Every candidate has a phone interview first and every candidate gets a response to the coding test. It's unusual to complete the test without e-mail back and forth with one of us for clarification (which is part of the test).
>And as for "Tic Tac Toe" as ANY sort of effective test topic,
First you wanted something that didn't take 12 hours, now you're complaining the test is too simple. You don't know what the company does, so how do you know the skills you need for a tictactoe solver aren't relevant?
>I did two coding tests in recent years ... in one, their feedback was "they didn't like it", and the other one I worked on for 12 hours because I believe in doing a good job and meeting the requirements, and I heard nothing at all back - not a single word.
Companies being bad at coding tests doesn't mean coding tests are bad. Companies are liable to ghost you at any point in the interview, that's nothing to do with a coding test.
Our interview process is on-site and normally takes 1/2 visits, it will involve probably 8 person-hours of our engineering time + management. So yes, we want to make sure that candidates are atleast vaguely able to solve practical problems before they come in.
There's almost never any science or systematic approach to evaluation.
How can you possibly evaluate a test without predefined criteria for evaluating that makes the result good and what makes it bad?
Whether or not the code is "good" is simply a matter of opinion in almost all cases.
Another down side to coding tests is that if the assessor knows less than you about some area of coding and sees something they don't recognise or understand, then they might well negatively view what in fact is a much better way of doing things.
And, much worse than anything else, you have a very tangible chance of spending 12 hours doing it and hearing nothing back at all.
And as for "Tic Tac Toe" as ANY sort of effective test topic, well, that's where you should pull out and find a different job to apply for, where they can at least formulate a relevant test topic and come up with tangible and defined criteria for measurement as well as a commitment from their end as to who will assess it, against what criteria and within what time frame and what feedback they will give you, if any. Ditch the Tic Tac Toe companies.
I did two coding tests in recent years ... in one, their feedback was "they didn't like it", and the other one I worked on for 12 hours because I believe in doing a good job and meeting the requirements, and I heard nothing at all back - not a single word.
Coding tests are, for the most part, bullshit.