Are the LEOs getting bonuses for raw number-of-arrests?
https://www.nbcnewyork.com/news/local/nypd-officers-arrest-q...
There may be a subconscious bias here... if you spend your entire year looking at murder suspects, you probably see the guy on the other side as just another scumbag... and probably want to make sure the victim gets justice (even if it's wrong)
Since you're not briefed on the crime and only given a list of tasks to perform (ie, locate pictures of a vehicle), there might be less of a bias here.
A good number are nazis, so if the software enables them to enjoy life better by harassing a few more black youth, that's enough of a reward.
Since this is a programmer-rich forum, let’s talk about bug fixing for a moment. When you are looking for a bug, it’s easy to convince yourself a number of things, like: “It’s a random glitch, just restart it and it won’t happen again.”, or “It’s this library which is buggy, it’ll be fixed when we upgrade it to the latest patch.”. Now, of course, we programmers are then punished mightily for our arrogance when the bug keeps happening. But what would happen to bug fixing if this never happened? If we could just declare that the bug was this or that, and move on?
This is exacerbated by deep-seated structural factors in the US, policing by consent is unknown, there is high income inequality and segregation, the level of training is pitiful, it's no surprise that Amber Guyger's instagram feed looked the way it did. Of course those types would use any tool they are handed to have fun at the people's expense. The system selects for them.
But I think police are molded by their experiences, and made by various vicious cycles to become the stereotypical bad police. I described one such cycle above; here is another: http://tailsteak.com/archive.php?num=545 If this is correct, then anybody, even the most well-meaning angelic people, would, over time, become bad police, but anyone could still merely claim that all police are bad because they are “right-wing” (as if that automatically is a bad thing) or racist, and nothing could ever be solved.
When we are told that a prediction is from an automated system, most people are more likely to believe it and not subject it to the same skepticism and critical thinking that we do to predictions from humans. Even if there is a human-in-the-loop system that requires a human to confirm it, that's often just laundering the machine's prediction through a human. When we are primed by being told this is what The System predicted, we're more likely to override our normal processes. This is especially the case when The System has worked 99% of the time --- which is still a lot of false positives.
Although this can also be the case with priming from predictions or judgments by other humans. The Asch conformity experiments involved planting fake test subjects in a group activity where subjects had to give answers to a very obvious perception question, and the real test subject could hear everyone else's answers. The fake subjects went first and all gave the blatantly wrong answer. The percent of people who conform varies with how you run the experiment, but typically a majority of the subjects end up giving the obvious wrong answer that everyone else did: https://en.wikipedia.org/wiki/Asch_conformity_experiments