The classic (multi-decade problem repeated again, and again, and again...) is face detection. Face detection models continue to get trained by primarily white developers/researches who continue to use primarily white people as their training set. And then they're shocked when their model performs terribly for everyone who isn't of European decent.

This case is so pervasive that it gets taught in basically every university that teaches ML - and yet those students go on to repeat the exact problem they were educated against.

> Face detection models continue to get trained by primarily white developers/researches who continue to use primarily white people as their training set.

I guess you didn't watch Asian developments in face recognition closely in the last few years.

Let’s pause all ML research until we have 50% black representation!

Does the face recognition tech in China have big problems with european faces?

A lighthearted example: https://www.kotaku.co.uk/2017/01/12/how-we-accidentally-made...

It's even worse than that.

Assume you want to train an AI to recognize shops or buildings for example for a Google car.

Well if you do it in the US with skyscraper, in Europe, in Africa, Middle-East or Asia, you will get completely different results and biases.

Also, I don't see how anyone has the resources to compensate such a social bias, unless they plan to do a shooting trip in hard to access locations and try to justify such as "Seriously officer, the reason I take all those photos is to make sure my machine learning model is unbiaised so I don't classify shops in your country as shacks."

And if we forget human activity, even looking at nature, the fauna and flora are different between areas their color, how sparse they are, etc.

Last example, applying sparsity to human activity, what is actually a town in some country might be classified as a settlement for example due to bias.

In this case, ironically, it seems that an AI could incorrectly depixelate faces of suspects caught by surveillance cameras to white instead of black. Which would lead to a bias against white people.

Please take a look at the PULSE model card here:

https://thegradient.pub/content/images/2020/06/image.png

PULSE is published as an art project, not suitable for face recognition or upscaling. It can only generate 'imaginary faces'.

They didn't even train the GAN they were using, it was borrowed from another paper. They probably used StyleGAN because it was a nice high quality generator and they invented a novel way to use GANs so they needed a toy model to showcase their algo.

Agreed. I was only pointing out that even bias is a matter of bias: if white faces had been depixelated to black faces instead of the other way around, the authors could have still been accused of a racist bias (because of the hypothetical scenario I've described above).