My impression is exactly the opposite. It flags them as a company that cares about technical competence.
"If you want to screen for the absolute basics of rote capability that's what those various certified professional type credentials are for."
Their test screen for more than that. And of course those professional certifications are useless if you go beyond the very basic level, just like "CV-years of experience" are useless for determining competence.
The credential I look for when hiring is someone who has already done something remarkable.
I bet the tasks being sent out are most probably something in the vein of parsing a CSV file or (in case of Atlassian) writing a simple JSP or trivial XSLT or what have you. Not the best check for competence.
And yes, he actually mentions in the article this is an actual thing he does: have applicants learn new frameworks in order to apply. Sheesh. Who has time for that except the very desperate.
In fact, bringing in people to an interview right away without at least calling them IS like asking the truckie to drive a long distance just for the sake of it. I'd avoid that. But I would ask him to send me a photo of his current truck (if he claims he has one). I'd maybe also ask him to name five roads he'd avoid between his main destinations (say in SF, if he claims to be from SF). Someone who's actually a cab driver in NY would find that hard to answer. :-)
Because it flags you as a mill, uninterested in the applicant and likely a firm that treats development as a commodity. I'd be willing to bet that the 30% who didn't bother to respond to the challenges included some of your best applicants.
You'd never hire an accountant by insisting they prepare a quarterly for you before you ever met them and you wouldn't make prospective truck drivers arbitrarily drive hundreds of miles to an interview.
If you want to screen for the absolute basics of rote capability that's what those various certified professional type credentials are for. But just like your little tests, you'll be pre-screening out the best as well as the worst.