News for hackers. Stuff that matters.
Re:A.I. Doesn't Get Black Twitter by wyager on Saturday September 24, 2016 @04:10PM (#12572424)
> If the training corpus for a machine learning model contains stereotypes and biases, then the output of the model will reflect those prejudices.

Depends what you mean by "the corpus". Let's take the example of loan applications. Maybe the training data contains credit scores, biographical information, and personal essays about the applicants from loan officers.

Let's say loan officers tend to use more negative language when talking about black people.

Any decent machine learning system given racial data will not adopt those biases. In fact, it will counteract them. It will determine that e.g. the semantic content of the essays is an indicator of loan suitability, and it will also notice if it has to e.g. add something to the essay score for black people to make the best decisions.

As long as the output data is objective, competent ML systems will account for any bias in the input data.

> Should we attempt to detect prejudices in machine learning models

We don't have to. If the output examples we are training on are true, the ML algorithm won't adopt any incorrect biases.

This is literally the point of machine learning. You want to make the best possible decision. If your algorithm has incorrect human biases, you're losing money. ML naturally accounts for this stuff.

Re:A.I. Doesn't Get Black Twitter by gugagore on Saturday September 24, 2016 @04:28PM (#12572506)
You are only talking about supervised learning.

And the bias people are probably talking about is bias in the data, not bias in the "input", as you put it.

See http://arxiv.org/abs/1607.06520 for a counterexample to your point.

Re:A.I. Doesn't Get Black Twitter by wyager on Saturday September 24, 2016 @09:56PM (#12573506)
In that paper it sounds like the algorithm is finding true relationships that the author doesn't like. What's the problem with the algorithm?
Re:A.I. Doesn't Get Black Twitter by gugagore on Sunday September 25, 2016 @01:34AM (#12574120)
I don't understand what you mean by "the algorithm" or "true relationships".

Do you know what I mean? There's truth as in what the data says, and there's truth as in what's underlying before social processes " corrupt " the truth. This paper examines ways to get at the latent truth.

Do you think that's a useful endeavor? It's dismissive to say it has anything to do with what the "author likes".

Re:A.I. Doesn't Get Black Twitter by wyager on Sunday September 25, 2016 @11:06AM (#12575629)
> and there's truth as in what's underlying before social processes " corrupt " the truth.

Ah, so you're defining "truth" as "whatever I imagine the world would be like if society worked the way I wished it did". There's another word for that; "fantasy".

Re:A.I. Doesn't Get Black Twitter by gugagore on Sunday September 25, 2016 @07:40PM (#12578080)
I asked you to define what you mean by "true relationship". And you did not.
Re:A.I. Doesn't Get Black Twitter by leftpad on Saturday September 24, 2016 @06:43PM (#12572967)
> If the output examples we are training on are true, the ML algorithm won't adopt any incorrect biases.

This is rarely the case when working wtih real data, and thus inspecting whether our models are biased against protected classes is probably one of the most important things an ML practitioner should do.

Re:A.I. Doesn't Get Black Twitter by wyager on Saturday September 24, 2016 @10:03PM (#12573522)
We have a good understanding of the error profiles of most ML algorithms. We can put tight bounds on the difference between predictions and validation data. If ML algorithms make mistakes, it's usually due to noise or low-quality data, not the algorithm itself.

There is no reason an ML algorithm would be biased against a "protected class". It doesn't know what those are. It's possible that the algorithm will uncover a truth that you don't like, like that there as differences in risk profiles across race or gender, but that doesn't mean the algorithm is incorrectly biased. It just means that reality is at odds with how you might want it to be.

Re:A.I. Doesn't Get Black Twitter by leftpad on Saturday September 24, 2016 @11:38PM (#12573817)
Not talking about the algorithm. It's the data that are biased. The choice of algorithm just determines how interpretable those biases actually are.
Re:A.I. Doesn't Get Black Twitter by wyager on Sunday September 25, 2016 @12:22AM (#12573944)
> It's the data that are biased.

Which data are you referring to? In most cases, the training data isn't human-generated, and if it is, we usually want to match human behavior as close as possible.

Re:A.I. Doesn't Get Black Twitter by leftpad on Sunday September 25, 2016 @01:55AM (#12574177)
Virtually all data used to predict crimes or recidivism is fraught with human bias, for example. Not sure that we want to reproduce the bias of criminal justice system in any prediction problem involving this type of data.

Read anything written by Solon Barocas: http://solon.barocas.org/

Re:A.I. Doesn't Get Black Twitter by wyager on Sunday September 25, 2016 @11:11AM (#12575651)
How is recidivism data biased? I'm sure that the information gleaned from parole officers and cops might be biased, but as long as the ML system is trained on whether or not someone actually reverted to committing crimes, it should be able to detect bias on the part of P.O.s and other functionaries and give a more accurate determination as to someone's chances of recidivism.