From the article:

> “The cops looked at each other. I heard one say that ‘the computer must have gotten it wrong.’” Williams learned that in investigating a theft from the store, a facial recognition system had tagged his driver’s license photo as matching the surveillance image. But the next steps, where investigators first confirm the match, then seek more evidence for an arrest, were poorly done and Williams was brought in. He had to spend 30 hours in jail and post a $1,000 bond before he was freed.

This article just explained that someone who was clearly not the right person was still arrested by human police officers and charged even though those very same human police realized it was the wrong person.

The police wouldn't have even shown up at this random person's house if it weren't for the technology. They were the failsafe, and they failed too.

That's not the fault of the face recognition, though. And it would have happened in the same way if a human had misclassified the image.

Do they propose nobody should ever look at images of suspects and try to identify them?

> why not scan the database automatically (the evil, evil facial recognition), and then double check by humans?

Because people tend to think "The computer said this was right, therefore it is unequivocally the correct answer." This is obviously incorrect, but is how many laypeople view it, even if instructed otherwise.

Then people have to relearn, but it is not the fault of the algorithm.

False identifications by humans happen a lot, too. I think it used to be an especially big issue for black people.

Like if a black person commits a crime, and police does a lineup for a witness to identify the culprit, they would just point to the one black person in the lineup and think they did it. Surely computers can at least do as good as that, probably better.

In fact that's exactly what happened. The police realized it was the wrong guy and still brought him in, he still lose 30 hours and had to post bail.

I cant even imagine if two morons showed up to my house, saw that I was the wrong person, and then arrested me in front of my family anyway.

But then it can't possibly be the "people just believe computers" effect, because the article explicitly states the police realized the computer was wrong.

Then in this case it's the same as phishing emails. People know it's wrong, and yet still follow along anyways.

This is not necessarily a forgone conclusion, but occurs to often to dismiss.

Nothing indicates that it is specific to algorithmic facial recognition. Actual humans make mistakes identifying suspects all the time. At the very least, the article should address that and compare the reactions, to prove people are more likely to act despite knowing better when the id was made by a computer.

A reasonable point.