It's hard to grok how difficult and complex hiring good people is until you find yourself on the hiring side of the equation. The leaked Valve manual does offer some insight, but even that's barely scratching the surface.
1) I'm not that surprised. For example, I know that half the recruiters who call me are asking about positions for which I'm blatantly unqualified and which simply don't appeal to me. When I've carried through to an interview, this often becomes abundantly clear to both sides.
2) If you're only even interviewing such a tiny fraction of the many, many applicants you hear from, you simply don't get to claim there's an engineer shortage. There's a shortage of engineers you like.
If you only hire the top 1% of applicants, you will never hire more than 1% of the applicant pool... no matter how large it is. You'll just get the vast majority of the engineering population pissed-off at your hiring practices because you reject 99 resumes for every 1 you offer to while complaining of a shortage.
You get thousands of resumes, and you can't possibly call every one of them. So you try to find ways to build a funnel, hoping that your heuristics are weeding out the majority of those who are unsuitable for the position and keeping the majority of those who are suitable.
It's not that the company is being a dick, or that they don't value the candidate's time; it's simply that they can't possibly find each and every needle in the haystack, or find any at all unless they start filtering on SOMETHING.
So unless you have something compelling on your resume and cover letter that makes you stand out (not saying it has to be source code; just saying it has to be compelling), you're a lot more likely to be filtered out of the first funnel and never make it to the phone screen to have a dialog.