Did you read Thomas' "The Hiring Post" which was at the top of HN a couple days ago?
I would love to improve our hiring process, and include more realistic work samples as part of it.
However, I don't feel that a full-fledged work-sample test is realistic, especially if every employer started requiring it. Imagine applying to a few different jobs, and each requiring you to do 20 hours of learning of their material followed by some open ended realistic problem, that could take many hours to complete. If you were looking around at a few different employers, this would be a full-time job itself; some candidates would skip it because they could find a job which required less effort up front easily enough, some would simply not have the time because they are working a full-time job while doing it.
On the other hand, if there is a shared work-sample test like this one, that will be used by a number of employers, it means that it can be more in depth than any single employer could do, while simultaneously wasting less of the candidate's time as they do the one test, rather than one per potential employer.
So yes, in a way I am viewing this as a replacement for a technical phone screen, because I am interested in improvements to that process but haven't come across any potential improvements which seem viable, but this one sounds like it could have potential.
The "read the resume, read their existing code samples, and do a phone screen" approach covers a lot of ground in a little time, so while there are a large number of problems with it, it's pretty hard to improve on without spending a lot of the candidate's time to get up to speed on a more realistically sized problem, and a lot of good candidates aren't willing to devote that much time unless they really, really want to work for that particular employer. We're a small company, working in a niche business, so it's hard to get that kind of dedication from a single candidate.
Does a company need to be hiring 10 or more engineers to make it worth it to talk to you?
I'm at a small company, that has been trying to hire engineers for the last couple of years, and not having much success; we've hired a couple, had a couple take other offers, lost a few to attrition.
We would love to be able to get some candidates who have already passed a technical screen; less time for us, scheduling an hour with a bunch of candidates who don't pan out, less time for them, only focusing on one more in-depth set of problems rather than a whole bunch of different phone screens.
But we're not hiring 10+ developers any time soon; that would double the worldwide size of our dev team. We're looking for maybe 3 or so at this point.
Would you consider working with smaller companies as well?