> We need to be very careful when using AI systems to evaluate people in general -- not because they might be biased (which they might be), but because even a small bias, if used by virtually everyone, can be damning.

I don't think this even requires any bias.

Assume there's some loose ordering of who is or isn't a good hire, and every employer has their own fuzzy view of it. If you get slightly better or worse as a potential hire (pick up an extra degree, let your latest certification lapse, whatever), it gets somewhat easier or harder to get hired.

Now assume that same ordering, but all employers share the same view of it. I'd expect the divide between employable and not employable to be much sharper.

Well, I'd say that specific ordering is the bias. But I see what you mean. The bias is arbitrary, but still very real.

Also, we will of course have all kinds of attempts to "game" the system to get ahead. Optimizing (even more) for the metric. Degree mills, for instance.

It's strange to say it might be biased. Bias is absolutely impossible to avoid, especially with how today's "AI" works.

You might be able to avoid it with a panel of AI, similar to how we try to avoid it by using panels of humans, but even that turns out to be contentious and not surefire.

I have feeling with AI it'll be even worse, since folks / companies can pass the buck (similar to how health insurance companies are now using it to deny folks).

> * Bias is absolutely impossible to avoid, especially with how today's "AI" works.*

Unless you're taking the "there are multiple mathematically incompatible ways to define bias" view of the topic, just do what's already known best practice for high-bureaucracy human review. Which is too define an overly-pedantic standard rubric.

Everyone knows there is bias. The problem this article highlights is that by delegating screening and human judgment to a few AI vendors those vendors will bias all employers in the same way.

If you want to make meaningful change in this avenue you really can't use words like "bias" or "systemic" because anywhere from 49-51% of the population will immediately shut down upon hearing that. Someone can argue (and many do to varying levels of success) that systemic bias doesn't exist, which means this doesn't exist, which means there no problem.

However, "this AI model can decide that some subset of people, perhaps random, perhaps not, are simply not hirable for any job" makes sense to most people regardless of political bent.

The problem with the term “systemic bias” is that it takes a word that’s about differential treatment and changes the subject to disparate outcomes.

For example, the article here shows disparate impact: that different percentages of applications are passed through the AI filter. But it doesn’t show differential treatment of otherwise identical applications based on race.

Capitulation is a bad counterpropaganda tactic, especially with terms that have well defined domain specific meanings.

Note that the OP uses "systemic rejection", while the paper does reference bias, it is in the precise meaning of the word.[0] And this is not targeted at the general public.

You may want to look into the 1990 GOPAC handout "Language: A Key Mechanism of Control" to understand why some groups would simply just weaponize any term that was substituted. Academic papers need to error on being precise, to be effective, not focused on handling the general public with kids gloves IMHO.

Edited to add, listen to Lee Atwater's 1981 Interview on the Southern Strategy for even more context.

[0] https://arxiv.org/pdf/2605.27371

Agreed. Humans are also biased, but our biases are different across a lot of socio-economic factors. So when we have different people in these positions, the biases become less bias-y.

But LLMs are statistical models. They are aggregating all biases into a general super bias. And they're all converging towards the same solutions.

It is also illegal

What law does it violate? It's not even clear to me what "it" you're referring to.

capitalists have never cared about that

"we will amplify biases and possibly completely lock out portions of the population."

A lot of the capitalists see that as a positive.