The problem is that the system isn’t falsifiable. If the model says someone won’t turn up to court they are locked in cells until their trial, how will you know whether it was right or not? If you randomly release people as a test of the model then this is firstly unfair but secondarily won’t necessarily reveal small scale problems with the model that only affect certain groups of people because the sample will be small.

This also ignores the large impact that going to jail tends to have on people’s lives. I don’t think it is reasonable to simply dismiss the likelihood of bias.