The shortage is an imaginary bi-product of two actual problems. One problem is bias and the second is talent gaps.

Bias is a factor in that many interviewers have no idea what they want from a technical perspective, or simply lack confidence themselves. When in doubt hire somebody exactly like yourself. This isn't objective or a valid representation of competence, but it is common.

Talent gaps apply when there are a surplus, plethora is a better term, of new developers and senior developers are purple unicorns. It takes time and lots of practice to transform a newb into a rockstar. Throwing money at the problem isn't a magic formula for providing extra time and practice. Years of employment experience isn't an indicator of quality either as that doesn't necessarily mean practice solving hard problems.

When you solve for bias suddenly there are a lot more competent candidates available. If you realize a single senior paired with several newbs accounts for the talent gap more quickly suddenly you can hire functional teams without false expectations.

I think google did a good job of this in the early days. They got the word out on the street that their interviews were notoriously difficult. That probably dissuaded a lot of the random applicants.

I also read somewhere that some big bank (Goldman) just has a very circuitous, lengthy application process and the main point of it is to weed out the spray and pray applicants.

i’m pretty certain google is the most applied to company in the world so not sure if that holds up

I think that really started around 2005 when they made a concerted effort to ramp up college hiring (and by extension, started advertising among the general public).

I knew someone who worked there in 2002. (Well, now having worked there, I know lots of people who worked there in 2002, but in 2002 I knew someone who worked there.) At the time, they were a small startup with a reputation for hiring only the best, the way you might think of Medallion or D.E Shaw now. There wasn't a perception that you could just walk in off the street and get a job like in The Internship movie.

It helps to keep the illusion of exclusivity the more people you turn away.

I think there probably is a tech shortage in terms of strong candidates. I've worked with some companies where there are people that are employed as "developers", but they can't tell you the simplest of basics; I don't consider myself a developer, yet my meager knowledge and understanding far exceeds the skills of some of these people. I see more of these types than you'd think could survive for long (much of my work is outside of SV and with non-tech companies). When these types enter the job market, the local maximum they've been in for some time tells them that they're developers and they think they can apply to some of these more sophisticated gigs. But there's a clear distinction between these people that mostly copied code from SO and elsewhere in the code base and got it to work vs. those that actually devised the methods they're using... and I think there are many more at this lower end of the spectrum than there are at the higher end: just the bar is so much lower.

As for companies innovating in this regard, it's kinda funny to think about it. When it first got started, I think LinkedIn was trying to be exactly this. Rather than encouraging people to add any and every person they ever thought about (or that asked), the idea was that you should only add those people that you know and trust. In this way, you could use your LinkedIn network as a network of trust for things like hiring or looking for work. That's changed I think both by LinkedIn wanting to have a large network of data to mine and by any number of participants that will add connections since they're really after a bulk network rather than a trusted network. In both cases the business case for such a network of trust doesn't seem to work well for the company providing the service and doesn't work well for the participants... so it breaks. Naturally, that's just one approach and there could be others, but I do think there have been efforts here.

> I've worked with some companies where there are people that are employed as "developers", but they can't tell you the simplest of basics; I don't consider myself a developer, yet my meager knowledge and understanding far exceeds the skills of some of these people.

Hypothesis: these people might not have knowledge, but they're very willing to schlep, and so—with enough effort and iteration—they can take a plan and turn it into working code.

And on the other hand, some people who are very "talented" on paper are nigh-unemployable because they just can't get things done. (They can easily tell someone else how to get the job done, but they have no drive or desire to schlep for themselves.)

Sure. I would agree, though at some point this approach runs out of steam. Larger problems tend to lead to failure and the technical debt eventually demands to be paid. A not insignificant portion of my work comes from sorting out the messes when such failures happens; I have a solid understanding of the business domains and the application types (I basically have a good product management type knowledge, what the applications -should- do).

The other side of this is that, at this level of developer, these are exactly the positions where offshoring makes sense. If you're going to get people whose primary talent is a willingness to schlep just adequately enough to be employable... well, I can find those kinds all over the world, and can probably do better in the skills category, for much less cash. So if I'm looking for domestic (I'm in the US) technical talent, I'm usually looking for those that are worth the higher pay commanded in the US... which means I am absolutely trying to sort out the schlepers for the most part. Naturally, offshoring has its own difficulties regardless of why you're doing it; so there are times when schleppers (I like that term :-) ) are sufficient, but at some point that strategy changes (growth, etc.)

And adding on to this, these people are understandably most of the people looking for work.

I've literally seen engineers with 20 years of experience grind for 45 minutes on FizzBuzz, not able to make any progress.

> I’m assuming that candidates are just applying randomly to any positions that are available

Yep, that's it. The same cost-benefits dynamics as spam mail.