Re:Google is investigating the actions of another top AI eth...
by visarga on Wednesday January 20, 2021 @08:09AM
(#25845959)
Of course I was aware of her Gender Shades paper. It's a small bias evaluation benchmark dataset.
> We developed the Pilot Parliaments Benchmark (PPB) to achieve better intersectional representation on the basis of genderand skin type. PPB consists of 1270 individuals from three African countries (Rwanda, Senegal, South Africa) and three European countries (Ice-land, Finland, Sweden) selected for gender parity in the national parliaments.
A dataset of 1270 images is hardly a breakthrough, the kind I expect to see in a small university project. But it doesn't lead to better models because it's not nearly large enough to train on. What it can do is rate existing models. Basically - useful to critique, not to improve.
A small nitpick: why just two races in a de-biasing dataset? Where are the Asians?
She routinely offered suggested methods, experiments, and even new datasets [1] that fixed what she saw as wrong. She does have a fair bit of practical ML experience as shown in the computer vision papers in her publication list.
1. Buolamwini, Joy, and Timnit Gebru. "Gender shades: Intersectional accuracy disparities in commercial gender classification." Conference on fairness, accountability and transparency. 2018.