I can think of at least one trick that might work. Build a classifier capable of identifying race from the data, and then reverse its gradient to remove its signals. This would deracialize a dataset, so that there is no detectable latent information in it.
However, whether or not my particular solution works is somewhat beside the point. It's an engineering challenge to figure out how to do this, but that isn't an argument that ML is fundamentally bad for these purposes.
> For example, there are already attempts to use expert systems in the courts - to "scientifically" determine flight risk (and thus bail), and even chance of re-offence (and thus punishments). Those are trained on datasets of convictions. But we know that both conviction rates, and arrest rates, are themselves indicative of bias - e.g. some people get arrested more often for a crime, for which other people might get a verbal warning from police.
I totally agree that this is a problem. But it seems to me like the solution is more effort on fixing the biases, not reverting to human judgment. I think it's always important to make sure we understand what the baseline is we're comparing to. Humans are racist. Humans created the biases in these datasets. These algorithmic tools just bring those biases to the surface and make them quantifiable. It seems to me that that represents an incredible opportunity to actually fix the biases in a measurable way.
For example, there are already attempts to use expert systems in the courts - to "scientifically" determine flight risk (and thus bail), and even chance of re-offence (and thus punishments). Those are trained on datasets of convictions. But we know that both conviction rates, and arrest rates, are themselves indicative of bias - e.g. some people get arrested more often for a crime, for which other people might get a verbal warning from police.
So ML takes those biases, and entrenches them further. Worse yet, because people tend to believe that "computers cannot be biased", the fact that it's ML that's producing a specific result, is taken as prima facie evidence that said result is immune to criticism from that perspective. And with most advanced models, you can't really crack it open and make sense of what's inside - so you can't easily prove that it is biased.