News for hackers. Stuff that matters.
Re:Amazon scraps secret AI recruiting tool that showed bias ... by gsich on Wednesday October 10, 2018 @05:56PM (#18189273)
So they did. It should be obvious that if you don't want to include gender, then you have to sanitize gender-related data.
Re:Amazon scraps secret AI recruiting tool that showed bias ... by zaarn on Thursday October 11, 2018 @04:35AM (#18191812)
That's not as easy as one might think.

Machine learning generally doesn't have any prior opinions about things and will learn any possible correlation in the data.

It could for example discover that certain words or sentence structures used in the resume are more likely associated with bad candidates. Later you find out that <protected class> has a huge amount of people that use these certain words/structures while most other people don't.

And now the AI discriminates against them.

ML will pick up on any possible signal including noise.

Re:Amazon scraps secret AI recruiting tool that showed bias ... by s73v3r_ on Wednesday October 10, 2018 @07:08PM (#18189747)
More than that, though. Graduates of all-women colleges were also caught. If you're using school as a data point, that's extremely hard to sanitize.
Re:Amazon scraps secret AI recruiting tool that showed bias ... by gsich on Wednesday October 10, 2018 @09:18PM (#18190361)
Then what is the purpose of this? At some point you want this thing to "discriminate" (or "select", if this is a better word) people based on what they have done in life. Which is not negative per se.
Re:Amazon scraps secret AI recruiting tool that showed bias ... by s73v3r_ on Thursday October 11, 2018 @12:29PM (#18194874)
But you don't want it to select based on gender.
Re:Amazon scraps secret AI recruiting tool that showed bias ... by gsich on Thursday October 11, 2018 @06:20PM (#18197482)
Would it though? A school name is essentially just that, no gender information there, even with the "women" prefix. If you discriminate other schools, you can do it too with those. FWIW there could be a difference in performance which the ML finds.
Re:Amazon scraps secret AI recruiting tool that showed bias ... by s73v3r_ on Thursday October 11, 2018 @07:38PM (#18197921)
It would. Just because it's not explicitly looking for a "W" in the gender field doesn't mean it's not able to determine gender and discriminate based on that. The article and the discussion is all about how these things, despite not explicitly being told to discriminate based on gender, or race, or any number of factors, can still end up gathering a set of factors that are more prevalent among those groups, and discriminate against those people all the same.
Re:Amazon scraps secret AI recruiting tool that showed bias ... by gsich on Thursday October 11, 2018 @09:17PM (#18198384)
>despite not explicitly being told to discriminate based on gender, or race, or any number of factors

Then this is completeley useless. You want this "AI" to discriminate based on a number of things. That's the whole point. You want to find people that can work for you. If a specific school or title is a bad indicator (based on what you hired now), then it just is that.