You miss the point. We don't have a computer process that establishes factors such as the level of remorse expressed by the offender or their claimed motive, whether their misinterpretation of the relevant law is plausible or not, or the magnitude of the harm caused to the victim or whether the plaintiff's claim of monetary losses suffered is plausible or not so the "algorithm" is a thin veneer over a human determining which factors are and aren't relevant... so you get exactly the same level of bias before plus noise from the human judge making these determinations not knowing exactly how the algorithm works and a false claim of greater impartiality.

It's probably even worse if you try to pull the human out the loop altogether and throw an ML black box at the testimony, which doesn't understand the actual situation the defendant describes at all but does pick up the statistical association between African American vernacular words used and higher sentences in its training corpus. Much easier to train a model that picks up the prejudice in sentencing factors than the situational nuance...

I think the point is that we are currently using opaque, poorly trained and erratic ML (Meat Learning) algorithms. Oh sure the meatbox says that it's allocating N years for this, M years for that, but empirical studies have proven that this readout is nonsense.

Which kind of ML is easier to train to be non-racist? I don't know, but it's not obvious to me that meat is the winner here.