News for hackers. Stuff that matters.
Re:Training Computers to Find Future Criminals by cfallin on Tuesday July 19, 2016 @01:23AM (#12119899)
Given that ML is likely going to be used more and more -- even if not for criminal justice applications, at least in the private sector (credit scores/loan applications, insurance, etc) -- I wonder if there's a way to develop standardized "anti-discrimination standards" and then apply some regulation to any ML algorithm that makes a "life-altering decision" by some definition?

E.g. -- a "differential discrimination" test of sorts: altering any one variable of a "protected class" such as race, gender, religion, etc. does not change the answer. You would maybe want to pick a set of canonical test profiles among real people who differ only on one axis (as closely as possible), rather than just take a test point and alter one axis directly, because you'd want all the relevant correlations (ZIP code vs wealth, etc) to remain authentic. The end result would be a set of "equivalence classes": sets of human profiles who must be considered equivalent on all relevant life-altering judgments.

Or perhaps a "unit test"-like approach: similar to how one creates a unit test for each bug one fixes, create a "criminal justice ML algorithm test suite" with canonical profiles and their results: you must judge this person to likely not re-offend, you must judge that person as a high risk, you must judge this person worthy of a home loan for $X, etc. Sort of like a body of case law. I guess the risk is overfitting -- so maybe this data set is held in trust by some regulatory agency and not revealed.

People have probably thought about this and I haven't read your links -- is building a test data set and building regulations around it something that's considered?

Re:Training Computers to Find Future Criminals by Smerity on Tuesday July 19, 2016 @03:14AM (#12120292)
The European Union are likely to accelerate much of this type of research. They recently introduced regulations specifically targeting algorithmic decision-making and a "right to explanation" for automated decisions.

There's a Wired article[1] on the general scene and a paper that investigates the impact it may have on the industry[2].

Regarding differential discrimination, check out these two papers[3][4] - they're very similar to your idea :)

[1]: http://www.wired.com/2016/07/artificial-intelligence-setting...

[2]: http://arxiv.org/abs/1606.08813v2

[3]: http://arxiv.org/abs/1511.05897

[4]: http://arxiv.org/abs/1412.3756