While I don't disagree our legal system is also racist, the racism pales in comparison to the current level of sexism.
"COMPAS offers separate norms for males, females, community and incarcerated populations"
And they call that "defensible decisions"! In other words, they want to make sure the software doesn't rock the boat. If there's a bias, they want to preserve it so it doesn't look like their algorithms are "weird".
Isn't that what judges do when they follow precedent?
When they follow precedent, they are not matching outcomes to outcomes to avoid looking wierd, they are applying what was previously established as the formal decision rule.
That's a very different thing that attempting to preserve biases that may not even reflect the formally-annoubced decision rules in the cases that produced them, but may represent silent deviations from the formal rules.
Not to mention you'd never get access to do the differentiation, as it would rely on access to the proprietary process; the company would have to be compelled to cooperate.
- Write the algorithm out.
- Extract all the decision points.
- Pay legal interns to collect all the data, apply the data to the decision points.
- Produce a report.
- Produce a recommendation.
- The judge sentences based on that process.
And now tell the defense that they have no right to any knowledge about the above process. Not the decision points, and not what data was fed to what decision, and not even what the data was.
If you can't release that information to the defense because of trade secrets, then, it seems to me, it's not appropriate as a sentencing tool.
Of course, IANAL.