It was discussed on HN a few months ago: https://news.ycombinator.com/item?id=23696427

Basically (to summarize a lot), his point was: an ML model is only as good as the data you feed it. If, say, the photos for your face recognition model are only of white men, then obviously the model will do well on white men, while (possibly) not doing as well on other races or genders. This is a statement of fact, and nothing controversial about it.

But she took offense to it, and started insulting him on Twitter. He got tired of defending himself, and just quit twitter.

That LeCunn needed to defend his own character out of such simple discussion shows everything that's wrong with the US' progressive culture.

She was never uncivil and she pointed out a series of issues, from what she believed was imprecise in his argument, to the societal issues that caused her to be viciously attacked by other people for engaging in the discussion.

Other people questioned whether data only can be accountable for the biases and were not subjected to the kind of vitriol she was. The fact I am a white cishet male shields me from a lot of misdirected anger.

Sorry that's ridiculous - as a observer with no vested interest it was * very * clear she was very uncivil in her tone, repeatedly.

Not just that, she never provided any links to tutorials/papers/etc that she had given on the topic, and when I looked into the one workshop (from memory) that someone else mentioned, it had literally nothing to do with the issue of dataset bias.

That episode gave me the impression she was more interested in drum-beating and axe-grinding than engaging constructively. I'm not surprised she seemed to be doing the same inside Google.

I agree, reading the article (https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...) and the thread I also don't see how one puts this on Timnit.

Coming back to the original tweet: if you changed the training data have more black people instead of white, would it perform the same but with inverted racial biases? Maybe? You really can't know without doing it. It might generate faces with a dark complexion but also big distortions or unrealistic colors. The original model doesn't just produce white people because of the training data, the hyperparameter tuning, perhaps the entire architecture, would have been modified until it produced acceptable outputs... using white training data. Ultimately the engineers and other humans behind the scenes are the arbiters of success, loss functions are chosen by humans, and swapping training data on the same model won't change those early decisions.

In another context I'm sure LeCun could have offered his somewhat reductionist take on this example of bias (something I might have done myself – reductionism flows in technologists' veins!) A discussion could have ensued, and everyone could have come out with a better understanding. A hot Twitter thread isn't where that will happen. Neither LeCun nor Timnit have the power to change what Twitter is. LeCun (reasonably!) doesn't like the nature of the discussion, and he leaves, and I think that's OK.

> Ultimately the engineers and other humans behind the scenes are the ultimate arbiters of success, loss functions are chosen by humans, and swapping training data on the same model won't change those early decisions

How would changing loss functions alter this? This makes no sense.

Hyperparameter tuning is done iteratively, and used get the best score on the test dataset. Do you really believe the engineers hand picked examples and purposefully trained it so that looked more "white" and disregarded the test scores?

Training/test data is 100% the cause for this bias.

The goal is to get pleasing face reconstruction, everything follows from that. There's no objectively correct loss function, it's something that is selected because it has a good effect.

A simple example could be sensitivity to color differences: with lighter skin tones there is a fairly large difference between eyebrows and eye features and the person's forehead or cheek. Someone with a pretty dark complexion might have features that are distinguished in a very compressed set of colors. Depending on the loss function a dark complexion face could be essentially flat and featureless.

This could be fixed of course... but it requires changing the loss function.

Learning conditional mean vs conditional median for superresolution might be differently affected by a large chunk of outgroup. That was what I remember folks talking about during this twitter feud. Or, like with word embeddings, where people have added some balance penalties to the loss function to make it unbiased in the presence of biased data. Dataset bias caused this, but dataset bias can sometimes be well mitigated.

I agree with you on the principles & morals, but frankly this reads as scolding - all he did was explain how this can be avoided, which is good context to provide to the public. There isn't some hidden message there where he's "dismissing concerns people have with AI" or he's facilitating "outcomes that could be used for evil or with racist intents"

His tweet doesn't rule out the possibility that he secretly has a malicious set of incentives, just like your handle doesn't rule out that you're a Communist spy sent to fracture the American psyche. Yet, its ridiculous. I feel like a lot of the world in 2020 is for some reason a hostile filter is applied ~99% of the time on the internet, and you _rarely_ see that in personal interactions in real life.