News for hackers. Stuff that matters.
Re:Awful-AI: A curated list to track current scary usages of AI by convery on Tuesday October 06, 2020 @03:42PM (#24701560)
Ehh.. the discrimination part sounds a lot like: "we had an AI look at statistics and it reported the patterns we choose to ignore".
Re:Awful-AI: A curated list to track current scary usages of AI by curiousllama on Tuesday October 06, 2020 @04:41PM (#24702118)
Ah, yes, but that's not really the point, is it?

COMPAS points out an uncomfortable trend ("violent crime is more prevalent in black communities...") but applies it in a way that is bad ("...and therefore black people should get longer criminal sentences for the same crime").

We both agree, I'm sure, that socioeconomics and race shouldn't meaningfully impact sentence length. We may also agree that it often does. What a lot of these models do is encode that outcome (that bias, some might say) in seemingly 'objective' systems. That reproduction of bias is why they're discriminatory - not because they look at crime stats, but because they apply them in a way that most people wouldn't.

Re:Awful-AI: A curated list to track current scary usages of AI by elisbce on Tuesday October 06, 2020 @09:21PM (#24704362)
You are deliberately twisting it. The crime stats says "violent crime is more prevalent in black communities...", but no AI/ML will produce the conclusion "...and therefore black people should get longer criminal sentences for the same crime". They are completely irrelevant. The AI model will simply conclude that race would be a factor when you are looking at crime rate, or posterior probability when a violent crime happens. And that is exactly accurate. What's wrong with that? That's a fact. Math doesn't lie nor does it have any agenda. There is nothing the AI is making up or adding up to. It is pure statistics and probability. It is the human beings that can't take the truth and the consequences.
Re:Awful-AI: A curated list to track current scary usages of AI by curiousllama on Tuesday October 06, 2020 @11:08PM (#24704946)
You're right, of course: "human beings cant take the truth and consequences." "That is exactly accurate [...] That's a fact" - both true. I think the issue is less truth and more action, as in "I believe it to be immoral to act upon that fact in manner X."

I can come up with a dozen logical statements that are equally correct and immoral to act upon in certain ways. If an ML algorithm had come up with them, they would be no less correct and no less immoral. For example, "If a woman applicant is pregnant, she will take maternity leave and be less productive year 1. Therefore, we should not hire her." <- likely correct, definitely immoral (imo).

COMPAS drew a number of objective conclusions and ties them to specific recommendations in its intended use: (1) if you're poor, you're more likely to recidivate. Therefore, you should get a longer sentence. (2) if you live in a black community, you're more likely to recidivate. Therefore, you should get a longer sentence (3) if you're related to a gang member (not just one yourself - even if your estranged brother joined a gang a decade ago), you're more likely to recidivate. Therefore, you should get a longer sentence.

Notice: the "you should get a longer sentence" bit is actually part of the tool. That was the intervention it was designed to suggest. There's a number of critiques of that connection (mathematical, legal, moral), but it's not mine, nor is it the "AI's"; it's Northpointe's. And ultimately that's the scandal & intrigue: the math is right logically, but even so, the tool's recommendation is wrong ethically.

Re:Awful-AI: A curated list to track current scary usages of AI by Hnrobert42 on Wednesday October 07, 2020 @11:09PM (#24715280)
The problem is that enforcement begets enforcement. Once one is entangled in the legal system, they are more likely to be called in for questioning, have their fingerprints possibly match a partial set left at a crime, have a possible DNA match turn into a warrant that leads to arrest on an unrelated crime. This is true because the web of laws, in the US at least, means that everyone breaks dozens of laws a day. So AI becomes a self-licking ice cream cone. Even worse, it is one that comes with a plausible argument of simply being an unbiased mathematical conclusion.

Of course math doesn’t have an agenda, but we aren’t talking about the actual algorithm having a human bias. We are talking about the application of math by humans to human problems using data sets curated by humans. Using math doesn’t purge the biases of all those humans from the system.

Re:Awful-AI: A curated list to track current scary usages of AI by jl6 on Tuesday October 06, 2020 @04:26PM (#24701976)
AIs produce correlation which gets sold as causation.
Re:Awful-AI: A curated list to track current scary usages of AI by 6gvONxR4sf7o on Tuesday October 06, 2020 @04:28PM (#24702002)
Which is then justified as causation we chose to ignore for political correctness.
Re:Awful-AI: A curated list to track current scary usages of AI by centimeter on Tuesday October 06, 2020 @04:33PM (#24702045)
A lot of the patterns these things pick up on are causal and we just don't like the fact that there are real causal networks that conflict with our religious/idealogical beliefs.
Re:Awful-AI: A curated list to track current scary usages of AI by lxe on Tuesday October 06, 2020 @04:46PM (#24702175)
The issue is using the correlation to make predictions. You're just perpetuating the trend instead of trying to change it.
Re:Awful-AI: A curated list to track current scary usages of AI by elisbce on Tuesday October 06, 2020 @09:29PM (#24704401)
It has nothing to do with AI/ML. If you see someone repeatedly stealing from your yard, you can't predict he would do it again with high probability, because it would perpetuate the trend instead of trying to change it? Why would the responsibility of change lie on the observers instead of the perpetrators? If he cares about his image, he can stop stealing and with time, people would conclude so.