Then, we can explicitly correct for that quantified bias. This loses the whole 'AI is mathematics, and mathematics is unbiased' appeal, because we are explicitly adding a bias to counter a pre-existing one.
My point is, this (or more advanced methods) is possible with AI. You can't do anything like this with cops.
the disagreement is whether or not we want a more powerful distributed system monitoring and enforcing the law
cops aren't perfect and often have biases but you can't just push some code to a server and make them do whatever you want them to, as you could potentially do with an AI-based system.
if bad people got control of the system they would be able to more effectively impose their will on the population
What I would do is, first list the characteristics that must not be used to make arrest decisions. Then, train an AI to recognise those characteristics. Afterwards, when training the crime predictor AI or whatever, use the first AI to scrub unwanted characteristics from the training data input. If it all works out right, the biases in the AI's decisions will reflect statistical differences between ethnicities, but won't be driven by them. And if it works out really well, you could use it in reverse a-la style transfer: what should the people of group X do differently to not be arrested so much.
To what degree? Based on first-order data, some populations really do commit more crime. However, we know that background/poverty is the real, underlying cause.
And even if we fed enough data to an algorithm to avoid that bias, avoiding the bias isn't enough. What you really want to do is tackle the cause. If the AI is a black box, you'd never know.
The problem with AI is almost always a lack of transparency, especially in the specific case, and not the general/overall case. Most training doesn't focus on this. Besides, the AI would limited by how good the bias tests we can come up with are.
Edit: For some use-cases, this general case optimisation is a feature, e.g. token customer service for cheap companies. Also much easier to control people who aren't that different.