News for hackers. Stuff that matters.
Re:Attacking discrimination with smarter machine learning by yummyfajitas on Monday November 21, 2016 @01:32PM (#13007822)
No, bogus correlations and inaccurate conclusions are explicitly NOT the computer science problem being worked on here. You are trying to frame the problem as algorithms wrongly classifying people. That's just a standard statistics problem (called "overfitting") and has nothing to do with fairness.

The problem being worked on here is "what if Armenians shouldn't get car loans because they don't pay them back as much as other groups?" I.e., algorithms rightly classifying people leads to results that we believe are "unfair".

This research is illustrating that you can't simultaneously have accuracy and fairness. You need to explicitly decide how much accuracy you are willing to give up to get fairness. I.e. it's computing the tradeoffs needed to evaluate the ethical question: how many Armenian deadbeats should you extend credit to in order to be "fair"?

Go play with the simulation to see. The various fairness criteria all achieve lower than maximal profits.

Re:Attacking discrimination with smarter machine learning by qb45 on Monday November 21, 2016 @02:25PM (#13008322)
I think what the research itself (as opposed to politics surrounding it) was about is summed up in the last paragraph:

A key result in the paper by Hardt, Price, and Srebro shows that—given essentially any scoring system—it's possible to efficiently find thresholds that meet any of these criteria.

The crucial thing is that this "credit score" doesn't actually predict defaults quite well, because blues who are 50% likely to default have higher score than oranges who are 50% likely to default. If you base decision on a simple credit score cutoff, the blue defaulters will screw you up because of their higher score. You can defend from this by tweaking cutoffs per-group to achieve profit maximization or various "fairness" goals. In a sense, it's an attempt at producing non-garbage out from garbage in.

And of course whether real-world implementations of this idea are "good" or "bad" is another can of worms altogether and varies case by case. They probably could have came up with some better examples than credit scores and labeling people by colors.

Re:Attacking discrimination with smarter machine learning by yummyfajitas on Monday November 21, 2016 @02:31PM (#13008387)
Let me repeat what I said: Go play with the simulation to see. The various fairness criteria all achieve lower than maximal profits.

The best predictor is one which is explicitly discriminatory based on race: it takes both FICO score and race into account. There are worse predictors which also discriminate explicitly based on race, but in some "fair" way.

Finally, the worst predictor throws away directly relevant racial information.

This is not about garbage input data - that's not the problem being addressed here at all. The input data is perfectly fine. The problem is just that the input data says "race discrimination is the best way to make money".

Re:Attacking discrimination with smarter machine learning by qb45 on Tuesday November 22, 2016 @06:20AM (#13013275)
I think you are being deceived by your real-world presumptions about the fictional example somewhat unfortunately and misleadingly engineered by Google. From your previous post:

> The problem being worked on here is "what if Armenians shouldn't get car loans because they don't pay them back as much as other groups?" I.e., algorithms rightly classifying people leads to results that we believe are "unfair".

No. If you stop playing with simulation and look at the input data, Armenians don't pay back less. They pay exactly as much as Iranians except that Iranians consistently have higher FICO scores because Iran infiltrated FICO with their suckxnet(TM) worm which replaces R binaries with hacked versions. Or something like that :)

The general abstract idea is: you have some input "score" which is know to inaccurately predict the outcome, use the input score and measurements of its biases to produce more accurate prediction than naive threshold classifier would.

And yes, the other guys talking about women's healthcare costing more or men causing more traffic accidents got it wrong too. I somewhat arbitrarily responded to you because you said something about "the real computer science problem being worked on here" and then continued to talk about other things like everybody else.

Re:Attacking discrimination with smarter machine learning by yummyfajitas on Tuesday November 22, 2016 @07:40AM (#13013642)
The general abstract idea is: you have some input "score" which is know to inaccurately predict the outcome, use the input score and measurements of its biases to produce more accurate prediction than naive threshold classifier would.

Why don't you quote the place in the paper where they make accuracy go up, fix overfitting, or build an improved risk score? Or even just quote a place in the paper where the risk score is treated as anything other than an accurate black box?

They pay exactly as much as Iranians except that Iranians consistently have higher FICO scores because Iran infiltrated FICO with their suckxnet(TM) worm which replaces R binaries with hacked versions.

In the example provided in the paper (see Fig 7) that's explicitly NOT true. Blacks pay back their loans a lot less than asians/whites holding FICO fixed. For example, at a FICO score of 500, blacks pay back their loans 10% of the time while Asians do about 40%.

I.e., blacks have consistently lower FICO scores because they don't pay back their loans. Further, FICO score is biased in favor of blacks. If we made it more accurate we'd be actively discriminating against blacks. For example, a black person with a financial situation reflecting a FICO of 500 would have their FICO score lowered to approx 450 to reflect their higher default rate.

Did you even read the paper, or the linked article?

Re:Attacking discrimination with smarter machine learning by qb45 on Tuesday November 22, 2016 @01:32PM (#13016260)
OK, I see what's the problem. By "higher score" I meant "higher score than the other group for given default risk", not "higher score than the other group on average". So, by my nomenclature, fig. 7.1 shows Blacks having "higher" (call it inflated) FICO scores because for every N an N% risk Black scores higher than an N% risk Asian. At the same time fig. 7.2 shows Blacks having lower scores in general, which must be because they default more, as you observed.

I think we can agree that using the same cutoff for both races gives less accuracy than using higher cutoff for Blacks and lower for Asians, for some values of "higher" and "lower". You are right that "race discrimination is the best way to make money", but at the same time I think I'm right that this happens because FICO score already is racist to begin with, which is the "garbage in" I talked about.

Re:Attacking discrimination with smarter machine learning by yummyfajitas on Tuesday November 22, 2016 @02:59PM (#13017102)
Yes, blacks have lower average FICO and FICO is also biased in their favor.

But this paper is NOT about correcting that bias. If it was then it would be a pretty short paper:

Abstract: Go use isotonic regression in scikit-learn, once for each group.

References: Some papers from the 1970's when isotonic regression was developed.

http://scikit-learn.org/stable/modules/calibration.html

...but at the same time I think I'm right that this happens because FICO score already is racist to begin with...

This is also incorrect. The conclusions of the paper would remain valid if FICO were calibrated identically for all groups.

Can I suggest reading the paper and working through the math?

Re:Attacking discrimination with smarter machine learning by qb45 on Friday November 25, 2016 @05:07AM (#13035576)
> This is also incorrect. The conclusions of the paper would remain valid if FICO were calibrated identically for all groups.

I guess you are right, this was just the first disparity I noticed in the orange/blue example and I got fixated on it. And yes, I haven't read the whole thing and my maths may be a bit rusty nowadays :)

Now I see that the problem they attempt to solve with "equal opportunity" is FICO's (in)ability to fish reliable borrowers out of the whole population. Currently, FICO identifies a small number of reliable Black borrowers whom they give high scores, plus there are many Blacks who would pay back diluted in a sea of unreliable borrowers with low scores. While in, say, Asians, the ratio of reliable borrowers who had been given high scores is higher (fig. 8).

I think the issue is a bit more nuanced than "algorithms rightly showing that fairness is opposite to profit".

In particular

> The best predictor is one which is explicitly discriminatory based on race: it takes both FICO score and race into account. [...] the worst predictor ["race blind"] throws away directly relevant racial information.

As I noted, this is a case of reverse bias directly compensating for FICO's bias. Max profit simply grants loans to all FICO score buckets whose default risk is sufficiently small to be worth it. If FICO vs risk was race-independent, "max profit" would use the same FICO threshold for every race and hence would be equivalent to "race blind".

Analogously, "max profit" could be equivalent to "equal opportunity" if FICO was better at finding Blacks who can pay and putting them in low risk buckets (high score) so that it becomes feasible and profitable for banks to grant them loans. I believe this is what authors meant by incentivising classifiers to improve accuracy and yes, I was wrong suggesting that they found a way to improve accuracy here by using the input classifier as a black box. Their ideas only compensate for the aforementioned race-biased score inflation, which isn't the entirety of the problem, and put some financial burden for some other mispredictions on banks/FICO to pressure them into getting their shit together.

I think trying to apply "equal opportunity" in the real world may indeed turn into handouts to people who can't pay, because <handwaving> it's possible that poor people are just hard to classify correctly and if certain ethnicities are poorer than others, they will appear to be given less opportunity even though actually it's simply poor people in general who are being given less opportunity. If FICO finds ways to classify poor Blacks better, it may turn out that applying the same solutions to other groups will improve their true positive rates too and hence "Black opportunity disadvantage" will stay.</handwaving>

But otoh, it also is possible that <handwaving> for some reasons Blacks are classified with less accuracy than Asians, which contributes to the lower overall true positive rate of the race.</handwaving> Authors appear to be assuming this possibility, though I don't think the distinction can be made from data presented in the paper alone.

TL;DR: I think you were oversimplifying things, and so was I :)

Re:Attacking discrimination with smarter machine learning by tptacek on Monday November 21, 2016 @03:22PM (#13008830)
"We proposed a fairness measure that accomplishes two important desiderata. First, it remedies the main conceptual shortcomings of demographic parity as a fairness notion. Second, it is fully aligned with the central goal of supervised machine learning, that is, to build higher accuracy classifiers."

Your dispute may be with the authors.

Re:Attacking discrimination with smarter machine learning by yummyfajitas on Monday November 21, 2016 @03:49PM (#13009068)
No, you are misunderstanding things. They aren't improving accuracy, reducing overfitting, or improving generalizability. If you think I'm wrong, feel free to cite the part of the paper where they claim to build a better FICO score (to go with their example).
Re:Attacking discrimination with smarter machine learning by argonaut on Monday November 21, 2016 @08:13PM (#13010818)
You're misreading the meaning of "fully aligned." If you read the 3 paragraphs preceding that sentence, the authors are referring to organization incentives being "aligned."