A.I. Systems Echo Biases They’re Fed, Putting Scientists on Guard

Re:A.I. Systems Echo Biases They’re Fed, Putting Scientists ... by tsimionescu on Wednesday November 13, 2019 @04:35PM (#21529863)

You're missing the point when asking the final question. Predicting crime rates is not the goal in itself (unless you're running some kind of crime rate bet).

If you're going to use a model trained on simple, biased data, to get, say, insurance estimates for a for-profit company, the model will probably successfully increase profits, so it was a good model.

On the other hand, if you're going to use the same model to help with sentencing, where your goal is to see equality and justice, then the model will do very badly, since it will punish many people for the community/skin they happened to be born in.

[ Parent | on HN ]

Re:A.I. Systems Echo Biases They’re Fed, Putting Scientists ... by manfredo on Wednesday November 13, 2019 @05:13PM (#21530217)

The problem is, some groups do commit crimes at higher rates than others. If we are engineering the model to produce outputs with no disparities when disparities do exist in the real world, then our model is going to be biased.

For instance, men commit more crimes than women. If we are building an AI that predicts risk of committing crime (say, estimating rates of recidivism) and we forcibly make it report equal rates between men and women then we will be creating a discriminatory system because it will either under report the risk of men or over report the risk of women in order to achieve parity. Engineering parity of outcome in the model when the real world outcomes have disparities necessarily results in bias.

[ Parent | on HN ]