When we are told that a prediction is from an automated system, most people are more likely to believe it and not subject it to the same skepticism and critical thinking that we do to predictions from humans. Even if there is a human-in-the-loop system that requires a human to confirm it, that's often just laundering the machine's prediction through a human. When we are primed by being told this is what The System predicted, we're more likely to override our normal processes. This is especially the case when The System has worked 99% of the time --- which is still a lot of false positives.

Although this can also be the case with priming from predictions or judgments by other humans. The Asch conformity experiments involved planting fake test subjects in a group activity where subjects had to give answers to a very obvious perception question, and the real test subject could hear everyone else's answers. The fake subjects went first and all gave the blatantly wrong answer. The percent of people who conform varies with how you run the experiment, but typically a majority of the subjects end up giving the obvious wrong answer that everyone else did: https://en.wikipedia.org/wiki/Asch_conformity_experiments

Are the LEOs getting bonuses for raw number-of-arrests?

https://www.nbcnewyork.com/news/local/nypd-officers-arrest-q...

There may be a subconscious bias here... if you spend your entire year looking at murder suspects, you probably see the guy on the other side as just another scumbag... and probably want to make sure the victim gets justice (even if it's wrong)

Since you're not briefed on the crime and only given a list of tasks to perform (ie, locate pictures of a vehicle), there might be less of a bias here.

A good number are nazis, so if the software enables them to enjoy life better by harassing a few more black youth, that's enough of a reward.

Since this is a programmer-rich forum, let’s talk about bug fixing for a moment. When you are looking for a bug, it’s easy to convince yourself a number of things, like: “It’s a random glitch, just restart it and it won’t happen again.”, or “It’s this library which is buggy, it’ll be fixed when we upgrade it to the latest patch.”. Now, of course, we programmers are then punished mightily for our arrogance when the bug keeps happening. But what would happen to bug fixing if this never happened? If we could just declare that the bug was this or that, and move on?

This is exacerbated by deep-seated structural factors in the US, policing by consent is unknown, there is high income inequality and segregation, the level of training is pitiful, it's no surprise that Amber Guyger's instagram feed looked the way it did. Of course those types would use any tool they are handed to have fun at the people's expense. The system selects for them.

But I think police are molded by their experiences, and made by various vicious cycles to become the stereotypical bad police. I described one such cycle above; here is another: http://tailsteak.com/archive.php?num=545 If this is correct, then anybody, even the most well-meaning angelic people, would, over time, become bad police, but anyone could still merely claim that all police are bad because they are “right-wing” (as if that automatically is a bad thing) or racist, and nothing could ever be solved.

Pretty much. They will investigate for alibis and if there are fingerprints/DNA they will do that test. But absent other physical evidence a single eyewitness is good enough. Unsurprisingly they often are wrong. Netflix's Innocence Project has an episode where this happened. A woman was raped and thought she saw her assailant some time later. He was subsequently convicted and eventually exonerated due to DNA evidence.

Practically speaking there is going to be at least something else pointing to an accused. Otherwise, they wouldn't be showing that picture to a witness. But once you're suspected the standard is the witness picking the correct photo out of a line up of 6. That's enough to be arrested and often enough to be convicted.

Witnesses are easy to address at trial, and if you're lucky enough to have a competent lawyer, it's often possible to neutralize them even if they are telling the truth.

Systems are harder to cross-examine. Once successfully litigated, prosecutors can use precedent to avoid any scrutiny of the methods, operational processes, or other aspects.

I use a jury I served on as an example -- the police found a person who was almost certainly guilty of running over a person in a park late at night. The key piece of information was when it happened due to surrounding events. The key witness was out of play because he was arrested for perjury -- evidence came out immediately before trial indicating that he was driving a car after stumbling out of a bar, and he had testified that he was a passenger to the grand jury. So they had to scramble to use traffic light cameras (which are synced to GPS/cellular time) to establish a window when the defendant was in a particular area. Because of the events, the defense was able to cross-examine someone from the camera vendor, who was ripped up on the stand -- evidence thrown out and case settled for a reckless driving misdemeanor.

That event, where the vendor was questioned (and in this case failed to satisfy the court) is pretty rare. It's often assumed that systems work perfectly, while they often do not. For example, DNA matching is sound science, but forensic labs sometimes have unsound practices and controls.

If police are under pressure to close cases, and prosecutors are incentivized towards collecting scalps, the natural outcome with automated systems is going to be people going to jail because of bullshit tech.

Something like, "99% match, statistically this person is one of 10,000 out a million people in our city who would match"

The worst part of this is that police are often very well, often more than software engineers, and this is the quality of work they put out.

The median salary for cops is more than $105,000 a year[1] and some cops can make over $250,000+ a year[2] with over time. They can retire with full benefits and a pension after only 20 years of working.

This level of laziness and ineptitude shouldn't be accepted from what are often the highest paid government workers on the books.

[1] https://www.nj.com/news/2017/05/how_much_is_the_median_cop_s...

[2] https://www.nj.com/somerset/2019/11/4-cops-in-this-nj-town-e...

From what I can tell 105k a year is the highest median police salary for any state in the country. Why bring up New Jersey specifically and act like it's representative? Either use the number for the whole country or the number for Michigan, which is where the events in the article took place (and where the median salary is less than half the NJ amount).

No, I'll use New Jersey because it's the state that I'm most familiar with. New York City, which is across a river from NJ and is the largest city in the US and has the largest police department, has similar rates for their police force.

You will find that police officers are some of highest paid government employees in towns, counties and states, and are the highest paid government employees in some cases.

In places where the median salary for police officers are lower, median salaries for software engineers are much lower, too.

My point is that police officers often make more than a lot of software engineers on this site do, and they're rewarded with benefits that are better than most engineers', along with a full pension after working for only 20 years. Yet we expect higher quality of work from junior engineers working on dating apps than we do the cops whose lazy investigative work costs people their freedom.

That's fine; you should say something like "In New Jersey, the median salary for cops is...". Stating only that "The median salary for cops is...", and then giving the number for your home state, which just happens to be the largest number in the country, in a conversation about Michigan specifically and the US broadly, is simply misleading. Yes, even though your sources make it clear it's NJ data. Your sources are supposed to back up your statements, not clarify their incompleteness.

I don't disagree with the point you're trying to make. I just saw a surprising statement, checked the source, and realized that the statement was missing some critical context. And not even context that would make sense, like the Michigan number vs. the US number. This is context that no reader could possibly infer from the conversation or your wording. Making arguments that way weakens positions, it doesn't strengthen them.

The cops don't need proof for an arrest. That's why the standard is called probable cause. The cops just need a reason, not proof. [0]

(I don't think a facial recognition match should be enough for probable cause either)

[0] https://www.law.cornell.edu/wex/probable_cause

I agree with this.

> Facial recognition is evidence, not proof.

I don't think facial recognition is even evidence. At most, it's filtration/highlighting.

Police should be assumed to not be rigorous in pursuit of their duties. They should be assumed to look for positive pieces of evidence, not exculpatory ones.

Given those two facts about the world, giving them more tools that are only capable of producing positive pieces[1] of evidence (within a confidence level below 100%) will result in more false arrests.

[1] These facial recognition systems point them at suspects - they never exonerate a pre-supposed suspect.

Or, you just suppose that to be true?

AIUI certainty of march is much higher for light-skinned people: do officers get data on match certainty?

Forgive my suspicion, but flagrant assumption of racism fuels societal division and strong evidence can avoid unnecessary animosities.

> In one especially notorious encounter, police in 2009 arrested a distinguished black Harvard professor, Henry Louis Gates Jr., at his own home in Cambridge, Mass., after a white woman called the police to report a possible break-in.

https://www.washingtonpost.com/national/public-outrage-legis...

Breonna Taylor was not only shot to death in her sleep, but was also not given critical aid for twenty minutes after they discovered she was injured:

> Mr. Walker told investigators that Ms. Taylor coughed and struggled to breathe for at least five minutes after she was shot, according to The Louisville Courier Journal. She received no medical attention for more than 20 minutes after she was struck, The Courier Journal reported, citing dispatch logs.

https://www.nytimes.com/article/breonna-taylor-police.html

That, I would say, is a "shortcutting" of police procedure. "Avoiding unnecessary animosity" is less important than facing the problem American society has with racism headlong. Given the evidence that we have, where is the data that it is not happening?

Wouldn't it be better to rely on statistics, rather than cherry-picked anecdotes? For example, the number killed by police, normalized to some difficult to fudge measure of crime incidence, such as the homicide rate?

How do those two examples measure up against the 406 whites and 259 blacks killed by police in 2019 [1]?

[1] https://mappingpoliceviolence.org/nationaltrends

Police brutality is a leading cause of deaths among young men, but especially so for young black men:

> Among men of all races, ages 25 to 29, police killings are the sixth-leading cause of death, according to a study led by Frank Edwards of Rutgers University, with a total annual mortality risk of 1.8 deaths per 100,000 people. Accidental death, a category that includes automotive accidents and drug overdoses, was the biggest cause at 76.6 deaths per 100,000, and followed by suicide (26.7), other homicides (22.0), heart disease (7.0), and cancer (6.3).

> [...]

> For a black man, the risk of being killed by a police officer is about 2.5 times higher than that of a white man. “Our models predict that about 1 in 1,000 black men and boys will be killed by police over the life course,” the authors write.

https://www.washingtonpost.com/business/2019/08/08/police-sh...

There are plenty of statistics if you look for them. Which is why, in my mind, the onus is on skeptics to prove that police brutality does not especially affect the black population.

https://mappingpoliceviolence.org/nationaltrends looks at whites and Hispanics separately. Non-hispanic whites are only 60% of US population: https://en.wikipedia.org/wiki/Demographics_of_the_United_Sta...

You also failed to normalize to the crime rate. That kind of analysis would lead you to conclude that the police are shockingly biased against males, to a far, far greater degree than against blacks.