News for hackers. Stuff that matters.
Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by joemaller1 on Thursday July 26, 2018 @09:42AM (#17617398)
Things I want to know:

- Show us the side-by-side images of the false-positives. Are the matches plausible?

- What is the demographic distribution of the mugshot database? If the data is disproportionately biased, then that bias would be reflected in the false-positives. A casual skimming of some mughot websites shows a potentially significant racial bias.

Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by jerf on Thursday July 26, 2018 @09:48AM (#17617451)
I have a ton of quibbles with the article and methodology, honestly. I suspect it's just straight-up so flawed as to be irrelevant to the discussion.

At the same time, they do still have the correct final conclusion. Dragnet facial recognition is a bad idea that will produce too many false positives to be of use. Or at the very least, dragnet facial recognition used by people who put too much faith in it is a bad idea. It's at most useful to produce flags for slightly more attention; on its own, I would not consider it anywhere close to enough to arrest someone on.

And it can serve as a demonstration of how a clumsily-put-together system can be used to produce bad results, so, if we correctly assume that there may be people bodging together a system in much the same way the ACLU did, well, hey, it's a valid result then!

Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by Kalium on Thursday July 26, 2018 @12:36PM (#17618761)
I really want to see their methodology. I'd love to replicate it and play with certainty levels to see what happens.
Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by sudhirj on Thursday July 26, 2018 @11:16AM (#17618107)
Whether the matches are plausible is completely besides the point. This isn't an evaluation of Amazon's recognition quality, it's a demonstration that the system is fallible and notes on the consequences.

Same thing re data setup. Not the point again. Even if law enforcement hires the world's best computer vision experts (fat chance, it'll go to cheapest contractor) this is a note that there can be mistakes that get people killed.

Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by petermcneeley on Thursday July 26, 2018 @11:37AM (#17618291)
What are the consequences?? How is this any different than the suspect sketches? This is a far worse situation and still quite minor https://www.cbc.ca/news/canada/british-columbia/bc-mother-ch...
Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by sudhirj on Thursday July 26, 2018 @12:21PM (#17618647)
What does the link have to do with sketches? It's about name similarity on the no fly list.

Regarding sketches, they're actually interesting because they force a human to exercise judgement, being ambiguous by nature. This is fine - any law enforcement officer stopping someone who looks like a sketch is going to give them the benefit of the doubt. On the other hand, if a computer tells an officer a person ahead is an 89% match with a known killer, the conversation will start with guns drawn, and that will greatly increase the likelyhood of things going south.

You really don't want to be scanned by a trigger happy cop having a bad day. God knows who you happen to look like from that angle.

Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by zo1 on Thursday July 26, 2018 @01:28PM (#17619188)
"[..]conversation will start with guns drawn, and that will greatly increase the likelyhood of things going south."

That is where the two sides of this argument diverge. First: That things "will start with guns drawn". And Second: That things are more likely to "go south" for an innocent individual incorrectly blamed by this.

Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by stevew20 on Thursday July 26, 2018 @05:14PM (#17621213)
Police have a poor history in the US of exercising trigger discipline. A fair amount of this is due to overly aggressive training that is designed to keep the officers as safe as possible, sacrificing safety of anyone perceived as a threat (real or not), but another large portion is caused by the culture of dominance and entitlement that is endemic to our law enforcement.

If police engage a target with a weapon already drawn, there is a long list of things that person can do that will get them shot, and a very short (and often unclear) list of actions that will keep them safe.

Source: 10 years PMO in the Marine Corps, 1 month SFPD (then quit because their training was so shitty)

Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by larkeith on Thursday July 26, 2018 @04:11PM (#17620537)
Regarding point two, I can't really see any argument that an encounter with a police officer is not more likely to go south if the officer is under the misapprehension that one is a potentially dangerous criminal; Police shootings are already not infrequent in the US, and adding the tension of greater perceived danger makes an already volatile situation worse - a sudden movement that might otherwise pass unnoticed is much more threatening from someone who has been misidentified as an armed gunman, for example, and is far more likely to elicit lethal response.
Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by pas on Thursday July 26, 2018 @03:40PM (#17620308)
I don't really understand the sides and arguments you are implying here. Could you spell it out please? (I'm not from the US, and have no relevant experience with cops, guns.)
Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by HeyLaughingBoy on Thursday July 26, 2018 @11:53AM (#17618423)
One major difference is that non-technical people tend to trust what a computer says far more than they should. People are familiar with sketches and can make judgments on how clearly they match a face. They have no similar frame of reference for computer face recognition.
Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by l9k on Thursday July 26, 2018 @03:48PM (#17620349)
But people are familiar with photographs. There is no way the software wouldn't show the picture that's been matched before an action (arrest...) is taken
Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by StanislavPetrov on Thursday July 26, 2018 @10:03PM (#17622934)
Because most people don't view sketches with the sort of religious reverence they do technology.
Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by zo1 on Thursday July 26, 2018 @01:26PM (#17619164)
I don't see how this is a problem, unless the police are feeding it blurry surveillance camera footage then claiming "success, it matched person X", which I doubt, as it would fall apart at the smallest of questioning.

For the most part, this sort of article makes things seem like there will be some sort of racial-based, dystopian police future where automated algorithms will target people and have them put into jail. I don't see that happening in the near future at all. The police departments' personnel currently using this probably LOOK at the matches and only then make a further call. Just like I would assume AFIS fingerprint hit results are double-checked by skilled operators after matching to reduce false-positives.

Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by mrweasel on Thursday July 26, 2018 @09:51AM (#17617466)
Depending on the technologi behind Amazons facial recognitions, there's very real chance that the software simply associated some black facial features with "being criminal". If that's the case we're back to craniometry in the 1910-1940.

Perhaps lawmakers should consider implementing rules to ban facial recognition software until the developers can prove some like 99.999% accuracy. Otherwise we end up harassing a large number of people, and the credibility of the technology will stuff. As it stand any lawyer will be able to argue that facial recognition isn't admisible as evidence, do to the high margin of error.

Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by Kalium on Thursday July 26, 2018 @02:56PM (#17619929)
They fed it a large number of mugshots. They didn't ask "is this person a criminal". They asked it "does this face match one of the faces you've seen before".

You're right that systems have previously learned to infer black -> criminal. It's absolutley possible that could have happened here! Given that not every one flagged is a person of color, there's clearly more at work.

Systems like Rekognition let a user specify what level of certainty the system has that it has found a match. In this instance, the system was told to use 80%. Rekognition can be told to use any number from 0 to 100. Your number of 99.999% could be used without any difficulty whatsoever! Since the ACLU doesn't seem to be sharing enough to reproduce the test, we can't tell what would happen.

Regulation based on accuracy is a wonderful idea! One well worth pursuing. With that said, "accuracy" is perhaps difficult to define here. How would you determine that? How accuracy is measured is highly sensitive to what you use as a dataset, and different test datasets can easily produce very different impressions of how common false positives and false negatives are.

Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by StanislavPetrov on Thursday July 26, 2018 @10:04PM (#17622941)
They didn't tell it to use 80%, 80% is the default setting. Anyone who thinks that many law enforcement agencies aren't going to use the default setting ought to go look at their routers and other hardware and see how they are configured.
Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by Kalium on Thursday July 26, 2018 @11:37PM (#17623260)
I understand where you're coming from. They didn't choose that number! It was chosen for them by Amazon! It's on Amazon for not actively forcing users to consider what level of confidence they want to use. Amazon can, should, and I think is perhaps even morally obligated to make customers actively choose a confidence level.

Yet, might there not be other factors to consider? Bear in mind that this isn't pushbutton easy off-the-shelf. Some actual work is required here. There's probably a batch of Python scripts sitting around that did the work. The most charitable assumption is that whoever did this at the ACLU didn't look at how the tool under examination should be used at all and missed everything about confidence levels in whatever few docs they did read. Under the ACM code of ethics, I am comfortable calling that an unethical degree of incompetence and carelessness.

A less charitable interpretation is that they knew and didn't care because it produced the result they wanted. This is similarly unethical, in my mind on a level with the sort of p-hacking prevalent in social sciences.

You're completely right! They just used the default. It's just perhaps possible that there might be other factors worth considering.

Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by hanselot on Thursday July 26, 2018 @09:44AM (#17617415)
How dare you question our great overlords the ACLU. They would never use statistics to find results favourable to their hypothesis.
Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by dang on Thursday July 26, 2018 @01:29PM (#17619196)
Maybe not, but please don't post unsubstantive comments here.
Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by otakucode on Thursday July 26, 2018 @10:03AM (#17617584)
When facial recognition fails, it usually looks pretty weird. Like it will identify faces in the knots and grain patterns of a wooden wall behind somebody. It's really a very ineffective technology and it's disturbing that it gets sold as something you could use to justify locking someone in a cage over. Machines with pareidolia are not good tools.
Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by zo1 on Thursday July 26, 2018 @01:31PM (#17619204)
When people speak of "facial recognition", they speak of faces being compared to other faces. What you're mentioning is face-detection. Are you honestly saying that police departments will have a "criminal database" filled with wooden walls and furniture?
Re:Amazon’s Face Recognition Falsely Matched 28 Members of C... by otakucode on Sunday July 29, 2018 @07:00AM (#17637451)
I'm saying that police departments DO have databases filled with images that have wooden walls and such. Not furniture usually, but photos are often taken in sub-optimal circumstances even when the police or others are trained to take the photos properly. I know this is the case because I've worked on biometric systems for the past 18 years, and I have personally worked on a system which was identifying almost every face to a mugshot taken against a wall with wooden paneling and the matching algorithm was keying in on it as a face rather than the human. I'm not speaking theoretically, this is how these systems fail.