> I don't follow. If the software has been unsuccessful, that seems like a good reason to remove it. If, additionally, it has been making decisions on racial grounds, that seems like another good reason to remove it.
I mistyped there. I meant to say "few rational people could argue AGAINST its removal".
I had originally edited my sentence partially and mangled the meaning entirely.
Racially biased could mean two things: despite not taking into account race the algorithm groups results racially at a statistically significant rate, or the algorithm is outright using race as a factor.
I assumed the former because it was not made clear in the article what was being talked about. Half the article is about things that may/may not use racial information (it's not said), and half the article is about things that do.
Are we talking about the same thing here?
Frequency of doctor visits seems a good indicator for determining how likely an insurance company will have to pay on a policy. If you want equal outcome, you end up setting a higher premium for everyone, and skip the underwriting altogether. The next part of the discussion would be subsidizing the higher premium for those that can't afford it. Just keep in mind that because there are fingers on the scale, fewer people can afford the "equalized" insurance than could afford the honestly evaluated coverage. (Prices are higher for everyone, so the pool of subscribers ends up being smaller if subscribers can opt out.)
It becomes a never-ending cycle of controls and impositions leading to collapse. Have the realistic underwriting, lower prices, and subsidize those (overall) lower premiums when needed.
You've been denied parol because someone with a name similar to you skipped bail is unfair regardless of race.
You've been denied parol because you come from a rough neighborhood is also unfair, regardless of race.
In fact, even if 99.9% of all people named X skipped bail, it would be unjust to deny somone bail because his name is X.
Well, no.
It's not enough to simply decline to input racial data into the machine-learning system. The system may discover a proxy for race, and make decisions based on that proxy. Name and address, for instance, may be enough to give a good idea as to someone's race.
We expect human judges and parole-boards, for example, to discount racial information, even if there are demonstrable correlations associated with race. If the machine-learning system is not built with that discounting in mind, it isn't going to do so of its own accord, as it was of course built to detect and exploit correlations.
> If it could be shown that there's low success in the software, yet it's relied on despite racial bias, then I would think few rational people could argue for its removal.
I don't follow. If the software has been unsuccessful, that seems like a good reason to remove it. If, additionally, it has been making decisions on racial grounds, that seems like another good reason to remove it.
> Race seems like a red-herring until it's shown that the metrics support poor performance of these systems.
As I hope I've made clear above, this isn't right. It's not just about performance, it's also about ensuring the system does not make decisions on the basis of an individual's race.