The thing that is not scarce from the recruiters' view is applicants or maybe even resumes. This is because anyone looking for a job can easily, due to the magic of the internet, apply at dozens or even hundreds of jobs. And applicants don't even have to apply for jobs any more. If you're on LinkedIn, they can come looking for you, and again, due to the magic of the internet, there are plenty of possible people to consider for a job.
So instead of artificial scarcity we have artificial abundance.
This problem seems to be analogous to the online dating problem. Technology has made one part of the problem far too easy and this causes big problems in other parts of the pipeline.
But by the time a candidate is past a phone screen that’s a validated lead. A company that was really desperate to hire wouldn’t treat validated leads so poorly. That they do means that they aren’t and we should disregard self-serving claims to the contrary.
They certainly shouldn't treat validated leads poorly, but we've seen ample evidence that they often do. Part of the problem may be (I'm less sure about this part) is that it is essentially the same people -- and same organization -- managing the overall hiring process through all stages of the funnel. They start with a shotgun process and they keep using a shotgun process in later stages when they absolutely shouldn't be.
It's better if you've got a prospective hiring manager inside the company that wants to hire you and is keeping an eye on the overall process -- this is what often happens whey you're recruited by employee referral. However, this just isn't an option at places like Google (and I think Facebook and Amazon as well) where you get hired into a global pool rather than hired into a specific team.
A few years back when I was hiring at Code for America, I redid things so that it was harder for the lazy applicant but easier for the serious one. I asked people to send me something resume-like (LinkedIn was fine), but to skip the cover letter and instead write a short answer to 1 of 5 questions: http://williampietri.com/writing/2015/slightly-less-awful-hi...
It was great. Anybody who did that got a call back from me, usually within 24 hours, just to chat a bit about what we each were looking for. Starting from that made it much easier for me to treat each person like an actual person.
Part A is understandable in that the standard hiring process is optimized for the common case: scarce jobs and abundant, interchangeable workers. That gets you bulk resume submission, minimal human interaction, and indifferent, slipshod processes. Part B is just jackassery, but it gets you things like brainteaser interview questions, high-stress interviews, and a lot of biases smuggled in under things like "culture fit".
That makes very little sense for software jobs, where it's the workers that are scarce. A few years back, I said, "Wait, we spend all this time optimizing our software for user experience; why don't we do the same for job applications?" I ended up with a significantly different process, one that was a little more work for us, but a way better applicant experience. I also shifted focus from finding flaws to looking to see people at their best, which is not only a better experience, but I think gets better people. I encourage everybody to apply their design thinking and user empathy skills to their own hiring process. You'll be pleasantly surprised.