It really explains a lot about the C-level people and their lack of engagement with work (for my company at least!); they don't seem to be doing their full jobs as, well, they aren't. I really need to figure out how to get on this carousel.
Kudos to you. You have worked out one of the fairest ways to gauge someone. May your tribe increase!
It always seems strange to me that people won't bat an eye at taking a day (or more) off work to come on-site for technical interviews, yet the idea of giving someone a 6-hour project do at home, at their leisure, is somehow a bridge too far.
In real-world hiring experience: I gave candidates a choice between on-site interviews and take-home projects for a while. Zero people chose on-site interviews when given the option, so I stopped offering it.
The bottom line is that interviewing requires time. If someone isn't willing to lift a finger or allocate a few hours to an interview, how are we supposed to gauge their competency? The "just trust me" model doesn't work at all.
One of the key differences is the company has "skin in the game" at an interview, but not in the 6-hour project.
When you interview, you know the company has included you in a shortlist and they will put time into getting to know you. For the time put in, you have a chance to pitch yourself, to at least get across your general abilities and personality, and some people will pay some attention. Also you can learn more about the company and the people you might work with.
When you get a take-home project, it's like the way spam is cheap: It takes almost no effort for the company to send the same project to a lot more candidates at low cost.
You do the work, but there's no guarantee the company will really assess it. Sometimes the company drops the ball and just "forgets" the application, or discards it the same way résumés are filtered: By skimming though 6-hour project submissions and casually throwing some of them away. It's not uncommon to not hear back anything from a company after sending the result of working on their project.
Some candidates will enjoy doing a project more than an interview, but that's not the only consideration in determining their value to the candidate.
Personal experience: I have never had an interview last more than an hour. I have also been hired twice on a "just trust me" model and it worked reasonably well.
Replacing a day of interviews with a 6 hour project sounds fair. That isn't how most companies use projects.
I can understand not wanting to do that if you're a full-time employee, but if you're a freelance developer actively looking for jobs, why would a paid project be worse than an unpaid phone screen?
Why would this exclude people with family obligations? When I say "narrowly-acoped project," I mean 5-10 hours of work. I've never gotten an offer for a dev role where the application interview process took less than 5 hours, and those were unpaid hours.
In other words, some visa holders in the US are very much allowed to work, but not contract work.
5-10 hours would be similar to a typical interview loop yes, and if that was the entirety of it then it's not more time consuming than the typical engineering interview.
The alternative is to have them take time off of their dayjob and travel on-site for technical interviews.
In the real world, candidates are greatly appreciative of take-home assignments that let them work in the comfort of their own home, on their own equipment, at times that work best for them.
I feel like this is the solution that values candidate time the most. They don't have to do an interview, and I'm paying them for their time immediately.
Is it really worth it to lose a day in your weekend for a couple hundred bucks and the hope of getting yet another assignment or a job?
And, if we're being really honest here, there's a ton of informal backchannel talking that goes on, so your interview "performance" regardless of medium is affected by that at many places. I think if you aren't actively working to make your interview process objective, it's being strongly affected by many biases and informal relationships.
That's all to say that on the balance, taking up "some" or "all" of a candidates time isn't that interesting of thing to think about because the process is so broken in so many other ways and this one doesn't have a huge impact.
I can't get that from a take home assignment.
One strategy that's been successful for me is skipping the technical screen and jumping to a narrowly-scoped paid assignment. If they do well, I give them a larger paid assignment, and we work our way up to regular work together. If they do poorly, I pay them for their time but tell them it's not a match.
I'll add the caveat that I've only used this to hire junior to mid-level developers for part-time or short-term work. I've never tested it on a full-time hire, and I imagine that would be more difficult.
I've never written in detail about hiring developers, but I've shared similar strategies for hiring writers,[0] illustrators,[1] and editors.[2]
[0] https://mtlynch.io/hiring-content-writers/
[1] https://mtlynch.io/how-to-hire-a-cartoonist/
[2] https://mtlynch.io/editor/