News for hackers. Stuff that matters.

A comment on Unable to Post Bail? You Will Pay for That for Many Years. Parent comment

Re:Unable to Post Bail? You Will Pay for That for Many Years by netsharc on Monday March 04, 2019 @04:48PM (#19305922)
But wouldn't algorithms be easy to examine? At the least you can test it with similar circumstances and modify the "race" variable and see if the results are consistent or different. Humans have many self-image protective layers when questioned (e.g. "I'm not racist, I have black friends!"), Bbut hopefully algorithms are more transparent...
[ Parent | on HN ]
Re:Unable to Post Bail? You Will Pay for That for Many Years by throwaway_boi on Monday March 04, 2019 @04:55PM (#19305982)
Race variable is often correlated with other data.

Look at red lining in real estate/banking. They marked black neighbourhoods with red boxes, white neighbourhoods with green. The bank would approve business and home loans depending on if you were in a red or green area. Algorithmically the banks did not engage in a racist practice but systemically they are.

[ Parent | on HN ]
Re:Unable to Post Bail? You Will Pay for That for Many Years by umvi on Monday March 04, 2019 @05:50PM (#19306463)
But did they mark red areas because they were black neighborhoods or because historically people in those neighborhoods defaulted on loans?

An algorithm as simple as "mark neighborhoods with high default rates as red, low default rates as green" would probably turn out to be "racist" even if it is also flagging poor white trailer parks with red.

There's no easy way to solve that

[ Parent | on HN ]
Re:Unable to Post Bail? You Will Pay for That for Many Years by ska on Monday March 04, 2019 @06:48PM (#19306852)
At least in the case of historic redlining (i.e. where the term came from), we don't have to wonder - there has been a ton of research done on this, and it was pretty clearly a racially motivated discriminatory practice. A fact which resulting in, among other measures, the fair housing act in the late 60s....
[ Parent | on HN ]
Re:Unable to Post Bail? You Will Pay for That for Many Years by candiodari on Monday March 04, 2019 @07:20PM (#19307027)
And it is very well known how to test and correct for that. In a human, however, it cannot be corrected. As you point out, it can be very hard to see what rules have to do with races.

I would argue that in some cases it's bloody hard to see that you're doing it yourself.

[ Parent | on HN ]
Re:Unable to Post Bail? You Will Pay for That for Many Years by philwelch on Monday March 04, 2019 @05:18PM (#19306155)
An algorithm can be racially biased, or have biases that significantly correlate with race, without actually having a "race" variable.
[ Parent | on HN ]
Re:Unable to Post Bail? You Will Pay for That for Many Years by int_19h on Monday March 04, 2019 @06:11PM (#19306594)
I think what he's suggesting is feeding the input data that differs only on that variable, and if the algorithm produces differing output, then it is by definition racist (and then you send the authors back to the drawing board to figure out why).
[ Parent | on HN ]
Re:Unable to Post Bail? You Will Pay for That for Many Years by philwelch on Monday March 04, 2019 @09:20PM (#19307590)
That's fair.
[ Parent | on HN ]
Re:Unable to Post Bail? You Will Pay for That for Many Years by andrewflnr on Monday March 04, 2019 @04:52PM (#19305961)
Machine learning models in particular are famously, fiendishly difficult to examine.
[ Parent | on HN ]
Re:Unable to Post Bail? You Will Pay for That for Many Years by presscast on Monday March 04, 2019 @05:00PM (#19306026)
While agree with your general sentiment, you ought to point out that this very much depends on the algorithm in question. Things like decision trees/forests are quite observable.
[ Parent | on HN ]
Re:Unable to Post Bail? You Will Pay for That for Many Years by JustSomeNobody on Monday March 04, 2019 @05:09PM (#19306083)
> But wouldn't algorithms be easy to examine?

Examine by who? And would they be trustworthy?

[ Parent | on HN ]