in the best case they will be heavily audited and will not be allowed to make important decisions.
in the worst case biases will be programmed into the systems decision making algorithms, amplifying the power of those controlling the systems
one of the best systems we've come up with for government relies on biased actors fighting each other and compromising to reach fair agreements. giving too much power to one of these branches of government is not a good thing, no matter how much inherent human bias it eliminates from the system
The same does not hold for humans. Detecting the bias (down to the person level) is very hard. Going on to remedy it is even harder. You can't (or at least shouldn't) reprogram people.
In other words, you can change a racist machine, but not a racist cop.
To what degree? Based on first-order data, some populations really do commit more crime. However, we know that background/poverty is the real, underlying cause.
And even if we fed enough data to an algorithm to avoid that bias, avoiding the bias isn't enough. What you really want to do is tackle the cause. If the AI is a black box, you'd never know.
The problem with AI is almost always a lack of transparency, especially in the specific case, and not the general/overall case. Most training doesn't focus on this. Besides, the AI would limited by how good the bias tests we can come up with are.
Edit: For some use-cases, this general case optimisation is a feature, e.g. token customer service for cheap companies. Also much easier to control people who aren't that different.
Then, we can explicitly correct for that quantified bias. This loses the whole 'AI is mathematics, and mathematics is unbiased' appeal, because we are explicitly adding a bias to counter a pre-existing one.
My point is, this (or more advanced methods) is possible with AI. You can't do anything like this with cops.
the disagreement is whether or not we want a more powerful distributed system monitoring and enforcing the law
cops aren't perfect and often have biases but you can't just push some code to a server and make them do whatever you want them to, as you could potentially do with an AI-based system.
if bad people got control of the system they would be able to more effectively impose their will on the population
What I would do is, first list the characteristics that must not be used to make arrest decisions. Then, train an AI to recognise those characteristics. Afterwards, when training the crime predictor AI or whatever, use the first AI to scrub unwanted characteristics from the training data input. If it all works out right, the biases in the AI's decisions will reflect statistical differences between ethnicities, but won't be driven by them. And if it works out really well, you could use it in reverse a-la style transfer: what should the people of group X do differently to not be arrested so much.
And people trust it more because "computers can't be racist"
This goes against the idea that 'machines are unbiased because math'. And it is a form of positive discrimination, which not everyone likes, but it is an easy fix.
Now we can automate the biases of people so broad-brush and incompetent in approach, that they would install an AI system that automatically charges people for jaywalking, without it also having been trained on the set of things that commonly exist on roads. Great.
>At least an automated system can be analyzed for bias and, when it's found, corrected.
That sounds expensive. Why do that when you can make something cheap that mostly works and just hit cancel should it accidentally target anyone important?
It's already been demonstrated that computers can racially profile people. Everything from consumer cameras that tell the users that people in the pictures don't have their eyes open, but in reality are Asian; to chatbots that turn into Nazi bullhorns.
You can at least train a bad cop to be better. Or fire them. How do you fire an AI into which an authoritarian government poured billions of dollars, years of work, and reams of press releases?
AI was made by people, and because of that will always be flawed. It's only in science fiction books that computers and algorithms are perfect and clean and rational.
We're walking on razors edge here, it only takes one "election gone wrong" and we might find ourselves in a nightmare with no escape.
In my opinion, that just undermines the point. Nothing would have changed if the Securitate was controlling a bunch of AI sensors and such rather than a bunch of police officers.
Humans aren't machines. The police of a dictatorship, at some point, will have to confront an order to gun down protesters and the like to preserve the regime. Even if they're ideologically committed, they might recoil the order to kill and let the regime fall.
https://www.theguardian.com/world/2011/mar/02/protests-how-d...
> And the use of violence by the regime under challenge, argue both Petersen and Sadiki, acts as a further crucial trigger for escalation by protesters against the regime. It is not simply because the state use of violence - whether in Bahrain or Romania, Egypt or Libya - acts as reminder of its brutal nature at a time when it is more vulnerable than it realises. It is also because the use of state violence confronts those still part of the state with a moral and strategic question: whether to tolerate the use of force and hope the regime survives, or peel away and join the opposition.
> "It is the other side of the story to that of those rebelling," says Petersen. "There are different considerations for those in the military, police or special forces, defined by their role. They are required to make a choice: whether they can switch sides and hope the people accept their new narrative in the new world after the regime, or stick by it to the end.
> "Whether you defect to the opposition depends on these differential kinds of moral calculus. Ordinary soldiers, for instance, have their own calculus. And if they decide not to fire on protesters that sends a signal to the higher-ups in the military. Then pretty soon you might see the interest of the military changing."
> Petersen, however, has one caveat. That this kind of negotiation in an organisation like the military does not necessarily hold true if there is the early and "crushing" use of violence.
On the other hand, you can update 900,000 computer processes in a relatively short time.
But to anyone complaining about how this is dystopian: compare it to human police officers who can racially profile and exhibit their prejudices to disproportionately target segments of the population, and which there is absolutely zero easy fix for, especially when we consider how prevalent unconscious bias is.
At least an automated system can be analyzed for bias and, when it's found, corrected. There's a lot to be said (in terms of justice) for fair and uniform enforcement of a law, assuming of course there is due process for appeals (and that the law is democratically legitimate in the first place).