I wonder how much effort has been made to avoid these systems blacklisting people for protected traits (sex, age, race, etc) or disabilities.

Probably very little.

Who is to say the benefit of this algorithm isn't doing such filtering in a nominally neutral way? What if it turned out 40 year old Latino women with vision loss are top performers and the algorithms just naturally selected for them?

This is unlikely considering algorithms act on the given training data set, not on outliers imagined by playing devil's advocate:

remember amazon's recruiting program https://www.theverge.com/2018/10/10/17958784/ai-recruiting-t...

Which would imply that people with "average ideal" resumes are overpriced and you can get a steal on people who, for instance, are unusually qualified in specific ways but lack some common HR filter like a bachelor's.

Or the algorithms are actually accurate and people who aren't "average ideal" candidates are lousy hires on average.

This imply that cost-cutting measures have not been applied to these algorithms, which they probably were. This also imply that algorithms are accurate.

Right. This is a problem that the market solves on its own. Anyone who doesn't want an efficient market is, by definition, a rent seeker. Rent seeking makes us all worse off.

That would make going against the grain and not using any would give a chance at a competitive advantage, both in being able to get talent at a discount because of perceived flaws but also in having a non-homogenous workforce who can think creatively and see what the rest don't.