News for hackers. Stuff that matters.
Re:Cameras That Can Spot Shoplifters Even Before They Steal? by gbrown on Wednesday April 24, 2019 @11:07AM (#19739097)
So "legal" is the threshold for "bad"? I disagree.

If the proportion of white shoplifters and black shoplifters being caught is different, AI will just scale up that disparity. If reality is biased, we should work to fix that: not make it more efficient.

Re:Cameras That Can Spot Shoplifters Even Before They Steal? by mc32 on Wednesday April 24, 2019 @11:12AM (#19739167)
I think if they want to target shoplifters, so long as their accuracy is within reasonable range of effectiveness, it’s okay. 90% of the Thai, 92% of AngloAmericans, 89% of Nigerians, 88% of South Americans, 87% of locals indeed processed were shoplifting, regardless of absolute numbers, this is okay.
Re:Cameras That Can Spot Shoplifters Even Before They Steal? by gbrown on Wednesday April 24, 2019 @11:53AM (#19739647)
I think your attitude is really, really dangerous, because it ignores the long term effects these kinds of things have on society. People don't even really understand how humans apply rules in biased ways - when you attache the words "AI", "Machine Learning", or "Statistics", even well educated people tend to just accept the magic truth telling black box at face value (for a small version of this effect, Google "precision bias").

To specifically address your point:

"so long as their accuracy is within reasonable range of effectiveness"

This is a far too narrow look at what accuracy means - that's why we usually look at sensitivity, specificity, false negative rate, and false positive rates, and why we break those rates up over important subgroups. To the example numbers you give, let's posit (without evidence) that your numbers are correct - they don't tell us what proportion of people visiting the store belong to each group (is the model just learning to identify ethnicity?), and they don't tell us what proportion of each group is really shoplifting.

In many areas (drug crimes come to mind), there's good evidence that different ethnic groups commit crimes at similar rates, but arrest rates are wildly, grotesquely disproportionate. Imagine that AI is applied to such a system - it learns, correctly, that black people are more likely to be arrested for particular crimes. It then targets black people, and the accuracy numbers look good on paper. In fact, because of the trust we have in technology, such a system might help normalize or justify this crazy preexisting disparity.

If you were building a statistical model to describe predictors of shoplifting, this kind of bias wouldn't make it past a cursory level of peer review. When you're slapping "AI" on a product and implementing it at a huge scale, nobody bats an eye.

Re:Cameras That Can Spot Shoplifters Even Before They Steal? by mc32 on Wednesday April 24, 2019 @11:59AM (#19739726)
I get the positive feedback loop issue. I’m saying if I’m looking at the data (which is blind a -it doesn’t classify people by ethnicity) but manually when looking at and labeling the data we find the above proportions as the effective proportions, then I think we’re okay.
Re:Cameras That Can Spot Shoplifters Even Before They Steal? by gbrown on Wednesday April 24, 2019 @12:09PM (#19739835)
> I’m saying if I’m looking at the data

My point is about the data generating mechanism - making inferences conditional on a bad selection procedure is a bad idea. It would be bad statistics, and it's bad ML. In this case, my argument is that it's also bad for society.