So I went in the tool and looked through the discard pile and found more diverse candidates that hadn't been presented to me. One looked amazing—over a decade of directly relevant experience, Oxford grad, etc. I asked the recruiter why she hadn't given me that resume and she said "She's overqualified, she's got 15-20 years experience."
I was livid and said "I don't care if they have 20 years experience, I want someone who is qualified!" We ended up hiring her and she's been absolutely wonderful, and is still on the team ~5 years later.
I don't think the recruiter was consciously discriminating against the candidate, but there were definite implicit biases that she wasn't aware of.
Implicit bias training is now required. I personally found it to be very helpful, and try and be aware of my own biases. I always hire the candidate that I think will be best for the job, but try and always find a diverse group of candidates to talk to. It's a better process, and makes for a better (and more diverse) team.
I've certainly known people who would entertain a move if a new position fell in their lap but wouldn't likely seek one out.
Also "I want too much money" is a thing if all you need is a low level tech and the applicant is an expert in the field. It is not worth paying for expertise that you will not need. For example I hired a guy to do some very boring, routine cleanup in some databases; if I would hire a real DBA and pay double, what for? In this case I pay for the work and result, not for the expertise.
But to be fair I always tell the applicants what is the situation and give them the choice. I know it is a very rare case in the industry.
The sad thing is someone like that would probably produce a better product thanks to their years of experience, and even possibly become a mentor to the rest of the team, but that's not the kind of thing recruiters think about.
I think this is the crux of a lot of it. You only need one lead/rockstar/ninja/10x* for every 10-20 'developers'. And, as you said, there is always the fear they will leave tomorrow for 2x the money...
*) I wanna see 'evil genius' next! :-)
I really disagree, ideally you'd have at least one in every team.
edit: point being, if you can muddle thru - mgmt generally doesn't see it as a 'need' just an 'expense'
That needs to become outdated thinking, since most people but particularly the younger ones, don't stick around any job for more than a year or two max regardless.
Also, if you've got someone with 20 years experience, and they're willing to accept the same pay as someone with 5 years experience... are they really willing? Or are they going to bail for more money at the first opportunity? If you hire them, you have to worry about how long they'll stick.
So "more than qualified" can be a real thing, not just a discrimination thing.