News for hackers. Stuff that matters.
[dead]
Re:Attacking discrimination with smarter machine learning by sctb on Monday November 21, 2016 @11:31AM (#13006637)
This is uncivil and unsubstantive to the point where it's simply a troll comment, and those aren't needed on Hacker News.

We detached this subthread from https://news.ycombinator.com/item?id=13005244 and marked it off-topic.

Re:Attacking discrimination with smarter machine learning by d--b on Monday November 21, 2016 @10:43AM (#13006190)
Race and ability to repay a loan, are - and this is scientifically proven - uncorrelated. What is correlated is your social status and your ability to repay a loan.

However, there is a problem because the population of the US is not uniform. Because of history, some races were less wealthy than others. So far, this makes sense I suppose.

Now, talking about machine learning. The big specificity of machine learning (when it is implemented with neural nets) is that it is trained on data only. The machine doesn't know about biology, and it doesn't know about history. Therefore, the model sort of converges to a place where effectively it "believes" from the data it has seen that there is a correlation between race and ability to repay a loan. So basically, the historical bias that is in the data translates into a logical bias. And so, because of history, you're less likely to get a loan if you're a black person than if you're a white person. That is what I call "amplifying the status quo"

Now, as we live in a society that holds equality of chances as one of its core values, I think it is good that people who work in machine learning look at these bias, and have a criticism towards the results of the algorithms in terms of societal impacts, and not only in terms of economical gain.

This has nothing to do with totalitarianism. Yes it is not libertarian, but we don't live in a libertarian world.

Re:Attacking discrimination with smarter machine learning by yummyfajitas on Monday November 21, 2016 @02:18PM (#13008254)
Can you cite the scientific proof? From what I've seen, race is correlated with default probability after taking other obvious factors into account (e.g. income, education, single motherhood).

Figure 7 of the underlying paper we are discussing shows exactly that - a white/asian person with a 700 FICO score has about a 10% chance of defaulting, while a black person with a 700 FICO has about a 15% chance of default. (Those numbers are eyeballed from the graph, might not be that accurate.)

I can't cite all the information I have on this, but I've found a blog post which analyzes similar data and derives similar relationships in publicly available data:

https://randomcriticalanalysis.wordpress.com/2015/11/22/on-t...

Re:Attacking discrimination with smarter machine learning by tptacek on Monday November 21, 2016 @09:23AM (#13005545)
No, I think he's more advocating for a world where we don't encode the literal status quo into computer models that make decisions for us going forward, which is something that nobody of any political stripe is likely to want.
Re:Attacking discrimination with smarter machine learning by ihsw on Monday November 21, 2016 @10:00AM (#13005837)
What's wrong with encoding the status quo? Why do we have to encode progressive extremism into computer models?

Being progressive for the sake of "going forward" sounds like misguided idealism. What do we do when progressive computer models lead us down a harmful path, do we simply play the typical liberal blame game and point fingers everywhere else while digging our head into sand?

Re:Attacking discrimination with smarter machine learning by tptacek on Monday November 21, 2016 @12:06PM (#13006941)
I'm not sure you're following my point. Conservatives should be no happier than liberals about the prospect of a deeply imperfect status quo being baked into future decisionmaking, because there's much about the status quo conservatives don't like either.

This isn't even remotely far-fetched. If you're a rural working class person, it's not at all hard to see how machine learning algorithms could adversely affect decisions about your insurance rates, loan acceptances, sentencing, the frequency with which your homes and businesses are inspected, and so on.

The idea that "discrimination" is somehow a progressive bugbear is --- well, I don't know how better to say this --- deeply stupid. Swirling eyes-within-eyes sticking out from blobs of fur inverted machine learning stupid.

Re:Attacking discrimination with smarter machine learning by ihsw on Monday November 21, 2016 @12:28PM (#13007139)
I do understand the point you're making -- my concern centers around deeply flawed models being used despite their shortcomings because their inherently progressive slant is sacrosanct. They would instead be held up and fingers pointed elsewhere, either due to lack of data or incorrect usage of the resulting advice derived from existing bodies of data (or something more sinister, eg: sabotage by opposition parties).

What I'm trying to say is correcting discrimination under the guise that it's unfair to certain sections of society is chasing your own tail out of boredom -- what's the point? To improve society, or improve usefulness of machine learning-derived data to make radical (but necessary) decisions? Do we maintain our objectivity and move away from "improving society" when it's found to be counter-productive, or do we keep going forward with improving society and hope things get better?

Trying to "break out of the status quo" to produce more a accurate representation of the world sounds great, I'm totally all for removing human biases, but my concern is that it'll be handled badly to mistakenly attempt to benefit certain classes of society (eg: repeat of subprime mortgage crises from 07-09 but much more difficult to diagnose).

Re:Attacking discrimination with smarter machine learning by tptacek on Monday November 21, 2016 @12:32PM (#13007170)
I'm not sure you understand my point, because you keep talking about "progressive slant", and I keep trying to explain to you that ML is going to be just as hard --- in fact, probably far harsher --- on "red state" voters.

Think of it in terms of survival bias. ML models that discriminate against the well-educated urban workers that design and deploy them are not going to make it to production if they obviously discriminate against their own designers. Those developers and "data scientists" are not going to notice if those same models happen to deny college admission to kids who took a year off after graduating to work at a factory to help pay for their siblings room and board.

Re:Attacking discrimination with smarter machine learning by ihsw on Monday November 21, 2016 @12:35PM (#13007194)
That is actually a good analogy. Thank you for that.
Re:Attacking discrimination with smarter machine learning by wehere on Monday November 21, 2016 @11:39AM (#13006708)
jesus christ
Re:Attacking discrimination with smarter machine learning by alvarosm on Monday November 21, 2016 @09:31AM (#13005616)
No, no, he's advocating for a dreadful, sinister world where a bunch of high priests strip you of your freedom to decide how you conduct business...

How much "discrimination" will be good enough in the end? I'll tell you what the endgame is: no discrimination at all, everybody gets a loan. So, in other words, no loans at all because nobody would conduct business in those conditions.

Re:Attacking discrimination with smarter machine learning by tptacek on Monday November 21, 2016 @09:36AM (#13005664)
First, I'm pretty sure he's not.

Second, there's no political orientation I can think of that is entirely comfortable with the status quo. Conservatives believe themselves to be discriminated against, just in a different set of circumstances. The underlying problem with ML is conceptual, not political.