Joel Spolsky explains why this does not imply what you think it implies.
http://www.inc.com/magazine/20070501/column-guest.html
tl;dr; Imagine 1 low quality developer who applies to 9 places and gets rejected. There's also 1 high quality developer who applies to 1 place and gets accepted. Look, 90% of applications have been rejected!
I have also hired people that had to pass insane interviews due to the company culture and watched them fail at being a developer. I know many good devs that cannot pass these interviews that would run circles around these super whiteboard problem solvers.
Offer them a stimulating job, and pay them lots of money, more stimulating than their current employer, and more money than their current employer.
There you go, glad I could help.
If there is this "floating population of applicants in your industry that apply for nearly every opening posted online" as Joel puts it, then I'll offer that the software job market is also flooded with a floating population of "dog jobs" that are uninteresting, low pay, crappy company/work conditions, etc. that never get filled without paid recruiters spamming thousands of people daily. When you search job boards, nearly 100% of the jobs are such. When you get another LinkedIn message, it's more than likely a dog job. Don't get me wrong, I like hearing from recruiters, and I appreciate the opportunity of getting a heads up about the kinds of jobs that are out there, but the pickins of GOOD JOBS, jobs that I would leave my employer for, are pretty slim.
Improve job quality and you'll improve your candidate pool. Pay bananas, get monkeys.
Right now I would love to be working on amazing challenging stuff for these companies. Instead I have to decide if i want to go gungho studying my ass off for whiteboard interviews...and if I am insanely lucky I will get offered a salary already in line with my current one at a company that doesn't have billions in the bank...like all these SV companies that are so convinced there is some shortage.
Consider a hiring filter which rejects 90% of good applicants and 99% of bad applicants. In such a world, consider a population of 10 good applicants. These 10 applicants will go on 100 interviews, be rejected on 90 of them and each will find a job. So in this world where every applicant finds a job, you still get 9/10 applications being rejected.
This seems to agree with your (anecdotal, and likely biased [1]) experience.
[1] "I got rejected after an interview, therefore hiring is broken" is a perennial topic on HN.
For my current job, I had a total of 2 phone conversations (first with a recruiter, second with the hiring manager) over the course of a week when they all of a sudden dropped an offer on me. I was a bit surprised by it -- "What? An offer already? Uh, I was expecting some sort of interview first?"
I actually nearly walked away at that point, but was... talked into accepting after a little back-and-forth on the salary.
But anyway, I just ignore cold-call/emails from recruiters anymore, because I know it will just end in wasted time (like getting rejected after 4 rounds of hazing and a weekend-stealing homework project) and stress/frustration I don't need. Most companies are obviously being overwhelmed with large numbers of high quality applicants for every open position ;)
This seems to agree with your (anecdotal, and likely biased [1]) experience.
Spolsky's observations are no less anecdotal and no more data-driven than the previous poster's experiences.
In a true labor shortage this would be completely unsustainable. Trying to justify rejecting 90% of your good applicants to avoid "false negatives" and extending that logic to say that there's a shortage... Obviously retarded if you think about it.
If you think it's retarded, try and work out the arithmetic yourself.
If this was true you would see it across professions and at every level of position, yet it's something fairly unique to software development.
I wouldn't disregard it so quickly if the source was more neutral.
Sorry, but Spolsky is normally extremely insightful and if you read the article, he makes many excellent points that are worth considering no matter which angle you're approaching the problem from.
It's a disservice to the brilliant visa holders I know as well as US citizens. it's extremely hard for them to get a job at a good company because so many of the visas are getting soaked up by those abusing the system.
If there was a huge engineer shortage you would see companies step in to provide developer training problems. This is clearly not happening.
What I'm disputing is this notion that there's no shortage of qualified developers. There absolutely is—that's why you see substantial salary growth for senior engineers.
What exactly do you think bootcamps are? They're outsourced developer training programs whose entire business model depends on the high tech salaries.
Tying up visas effects only to wage diminution is just a big mistake. But I can't wait to witness the effect of being a lot more selective on visas. We'll see!
The most immediate effects of fixing the visa system will be employers suddenly offering "on the job training" to fill these lowly skilled jobs. It's hard to see this as anything but a net positive for US workers
At least from my experience (I used to be a consultant) consulting jobs mostly attract people that can't get a job elsewhere. They generally include low pay, long hours, and a lot of client interaction.
It's no surprise to me that the nation's biggest software consultancies are largely considered the worst places to work as a software developer.
H1B's know to apply at these jobs because they have a much better chance of getting them. They have a better chance of getting them because these same companies are generally the ones abusing the system to hire software developers below market rates.
At most companies, the hiring manager and interviewers, who are predominantly other engineers, get to decide whether or not to hire a candidate. There simply isn't a mechanism for upper management to manipulate the process in the way you suggest.
I would say the opposite is true based solely on hiring practices. Look at Google for example, who hire around 1/5000 people that apply.
The shortage is artificially maintained because it works in companies favor. It makes engineers more willing to work for low salaries since they have to work so hard to get an offer, and makes it possible for them to import "talent" which is 95% of the time an excuse to import cheap foreign labor.
Look at what these H1B's are mostly used for. Yeah sometimes it's specialized jobs but usually just generic "developer" or "consultant". Is it really true that you need to hire an H1B because you can't find a "developer"?. The visa process is meant to be used for skills you can't find in the US, and with this in mind it's clearly failed its purpose