Yann: “ML systems are biased when data is biased"
Timnit: “I’m sick of this framing. Tired of it. Many people have tried to explain, many scholars. Listen to us. You can’t just reduce harms caused by ML to dataset bias. Even amidst of world wide protests people don’t hear our voices and try to learn from us, they assume they’re experts in everything. Let us lead her and you follow. Just listen. And learn from scholars like @ruha9 [Ruha Benjamin, Associate Professor of African American Studies at Princeton University]"
I don't like engaging this kind of discussion but have to call out the "I'm so tired" bit every time I see it.
In this case, the issue is that "it's the dataset" is attacking a straw man: nobody is accusing ML practitioners to be explicit racists actively conspiring for their models to be racially biased. Everyone understands that it is (often) a result of the underlying dataset.
But, the argument goes, understanding the genesis of such bias isn't enough to excuse it. At least not when those models are put to actual use in, say, law-enforcement contexts. Or credit agencies, or hiring decisions, or really anything.
If your dog happened to bite any red-haired kid it came across, it doesn't matter that you think red-haired kids are just as good as others, or that the dog was once mistreated by a red-haired kid. You're not going to allow the dog to get near any red-haired kids. And if you do, you'll face charges of at least negligence when some red-haired kid gets mauled.
To tie this back to AI, it would mean that as long as your models produce racially biased results, they simply aren't ready for deployment or publication. Go find better data until your work is no longer liable to inflict harm on anybody.
That's the standard for all other industries: if your car is shown to reliably kill pedestrians below a certain height in crash tests, you go back to the drawing board. Adding a sticker "don't operate car in the vicinity of children" isn't enough.
You can disagree with that reasoning (although you'd be wrong). What you can't do, as a leader in AI, is to pretend to have never heard of it, or that it is so obviously wrong that it warrants no reply whatsoever.
> have to call out the "I'm so tired" bit every time I see it.
so... I guess you're tired of it?
I'm not.
Many other rhetorical moves have a lot of politics to muddle through / implicit assumptions that can be disavowed in bad faith, etc. I don't like to engage in that kind of discussion at all.
But the "I'm so tired" bit is both condescending (and it takes... something... to condescend to Yan LeCunn) and evasive. The end result is authoritarian -- look man, this is decided and settled, you haven't kept up.
Calling this out is interesting because it evidences on the contrary that the discussion isn't settled at all, and that she's unable or unwilling to frame it in a convivial manner that's conducive to progress.
I sit out of conversations I'm tired about.
But isn’t what you described so vague and subjective, that you can apply it to pretty much anything?
IMHO it’s more honest to say “I sit out of conversations I don’t like”, which is an entirely fair and understandable decision to make :).
> You can disagree with that reasoning (although you'd be wrong). What you can't do, as a leader in AI, is to pretend to have never heard of it, or that it is so obviously wrong that it warrants no reply whatsoever.
There's a third option. You can just decide to not engage with the condescending prick at all.
We have one of those at work. His area of responsibility is a very cross functional one, and while it is important, he have a great deal of difficulty getting people to participate. Because participation usually means getting told you're an idiot. Loudly, forcefully, and with more words.
That seems to be the case here, only substituting "idiot" out for "racist".
Am I saying something controversial or micro-aggressive here? So be it.
I have the distinct impression that what they want is for discussions to become personal. Don't let them get to you.
Well, the disagreement is about publication. Yann LeCun himself said that "The consequences of bias are considerably more dire in a deployed product than in an academic paper."[1]
Why would it be wrong to publish research based on biased data?
I assume she thinks that Yann has enough intellectual capacity to improve situation of biased ML but doesnt execute. Further, my guess is that the reason why he doesnt execute is not transparent to her, hence frustrating.
Dr. Gebru, among others, expects and insists that they do go beyond that and try to get involved in the social consequences of tech. They refuse, ergo conflict.
Yann has no obligation to do so on Twitter, and shouldn't be called out by coworkers publicly regarding that decision, in my opinion.
Yann was talking about a specific model, she was talking about ML & society on a grand scale. Can't we talk about specific things now? Why did he have to apologize for it?
My feeling is that Timnit is using people, she guilt trips, humiliates them online, makes a big fuss and it's all to whip up support in her crusades. She wants to win the public opinion by any means, so she needs famous opponents to pile on, such as Yann and Dean (who have universal admiration in the field). She's always playing the victim. I find this kind of behavior toxic, it degrades dialogue.
Isn’t Gebru an “AI researcher”, after all?
Every practitioner already knows that these models behave according to the data they are trained on. The general public has no use for this explanation because they aren't the one deciding on the training data. So the only practical purpose is to offer some superficial causal mechanism that sounds like an explanation to that public, presented as an unchanging fact of nature, and distract from the equally true fact a biased model is only published or used iff it is trained on biased data and someone decides to go ahead and publish/use it nonetheless.
Yeah GP is modeling exactly the behavior they're railing against. Claiming that something is wrong without any practical insights for how that thing might be better.
Tiredness is probably coming from trying to educate people on Twitter, you can't do that because tomorrow a fresh batch will come and you have to start over.
Yeah, it sort of implies that, yes, you're angry about something, but actually have no constructive advice or solutions, other than to accuse someone of being a bad person or even a racist.
The "I'm so tired" bit is similar to the "I'm not here to teach you" bit. Maybe someone can't teach you because they actually have nothing to teach.
You of course know about this, or would now about this considering she is called an "AI researcher at Google" in the title of this tread. So the only lazy and un-constructive criticism is to assume she has not made any arguments beyond your one-line caricature of what you believe her to be.
> “I think that now a lot of people have understood that we need to have more diverse datasets, but unfortunately I felt like that’s kind of where the understanding has stopped. It’s like ‘let’s diversify our datasets. And that’s kind of ethics and fairness, right?’ But you can’t ignore social and structural problems.“
This seems to be saying that Yann LeCun should actually be addressing social and structural problems, rather that just applying neural networks to data sets.
https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...
I don’t know what the right answer is but plenty of institutions come up with ways they attempt to deal with systemic biases. For example, blind auditions for positions in orchestras (you only hear the person auditioning, you can’t see them). Used to not be a thing. Now it is.
Point being that when people are motivated they are able to come up with stuff.
What if you're a nuclear engineer that just wanna make the hot bit go hotter, and don't want to care about the nasty habit that radiation has of killing people?
Someone in another thread on this had IMO a great comment[1] summarizing the sentiment:
> If you are going to say "Well, garbage-in, garbage-out!", why do you keep putting the racist garbage in?
It's a collection of images uploaded by random flickr users with permissive licenses.
If you’re confused as to whom Lecun is, google also “lecun arxiv”.
“I’m not sad. I’m not sorry. I’m angry. And I’m tired”
It's not "commonly accepted" in the sense of throwing one's hands up, saying "there's just no way we could find enough pictures of black people for our AI to no longer confuse them with Apes" and move on.
This is a consequence of the fact that it's not possible to do generalization, and thus learning, without bias to begin with. What an AI researcher does when deciding which architecture is essentially tuning the bias in the generalization system in order to produce better results, which is not in and of itself bad, and is indeed unavoidable.
By bias, I mean really anything that means that the function g() is used over the function h(). In practice this can be anything from the seed for the random values, to the choice of activation function, to the architecture of the network, and so on.
If the architectures are selected for their performance on dataset y, then it is possible that even when trained on dataset x, they retain some of the bias of dataset y, because their generalization bias, which is architectural, was tuned for dataset y. This is, admittedly, a theoretical result, and it's not clear that it applies in every case, but it is a solid point.
For that reason the point that benchmarking against non-biased datasets is important was made, and thus that there is more to bias than the final dataset used for training during deployment. Therefore, it's not only a responsibility of the engineering community, but also of the scientific community.
I've posted it somewhere here already, but here's a nice paper that explains the point that there is more to bias than simply the dataset : http://www.cs.cmu.edu/~tom/pubs/NeedForBias_1980.pdf
Interestingly enough, this is true of all learning systems, including humans, and is a proof of why it's impossible to be completely unbiased in really anything.
In general I take these types of tropes to be cloaked (politically correct) way of saying "fuck x group".
Dataset bias is only one way this occurs, though. Even if you include more colored faces, you still may be taking pictures with a camera that doesn't capture as much contrast. Or even if you retrain, you may have chosen model structure to optimize behavior in a biased dataset. Or... if you're considering the cost of misidentification by a facial recognition system used by, say, police, you need to be sure that you take a perspective that applies to society as a whole and not to your own interactions with police.
It's perfectly reasonable to say "AI is worsening the experience of minorities in various ways" and to differ with responses that assume with slight additional care in curating datasets that the problem will be completely solved.
In France, since you have to be able to explain in plain words the algorithm that was used to help to make an administrative decision, it indirectly makes these black box learning systems illegal.
(And yet they probably already have been used by the police?)
Now, the naive response is that the engineer that implemented a system should take responsibility for any harm done by the system - and I guess that if these engineers could lose their license to use a computer and face jail time, they might also start to put pressure upstream on researchers to provide better systems ?
That statement is contained in a paper I linked that dates from 1980. I'm pretty sure it's factually correct, because I'm trying to make nothing but a demonstration that an idée reçue, that ML systems are only based due to their data, is incorrect.
There's really no cloaked message behind it, and I'm not sure why one would think there is (hence the disclaimer). I think it's a pretty uncharitable interpretation.
That's reasonable. I may have taken the initial text in the wrong context. Thanks for clarifying.
To expand on my prior comment of "Fuck x group", perhaps the tone of prior tweets by Timnit Gebru short-cut me mentally, namely this one: https://twitter.com/timnitGebru/status/1274809418475950080
> Even amidst of world wide protests people don’t hear our voices and try to learn from us, they assume they’re experts in everything. Let us lead her and you follow. Just listen. And learn from scholars like @ruha9 We even brought her to your house, your conference.
It strikes me as very unprofessional, and frankly I believe her anti-white bias shone through in that instance.
Racism is racism, even for "good" purposes.
Personally, I don't think either indicate racism. I see the link you tweeted as an excessively abrasive response at a dismissive and reductive understanding of an issue, bias in learning system, despite research old and new. At the same time, I see LeCun's dismissal as a genuinely held position in error.
To see the first as racist but the second as okay seems to be a case of being much more charitable to one side than the other, though I understand how it may look this way.
She says 'let us lead and you follow' -- sure...where? What are we concretely doing about the "in-built bias" problem?
“Let’s generate fake data with GANs to prevent our AI from discovering politically uncomfortable regularities!”
It’s ugly work but someone has to do it, right?
Indeed.
I'd even go one step further and bet that her whole career plan is ML activism. If you scan the email, all the key words are in there. Textbook agitprop.
I feel sorry for the next company that will hire her.
Even trivial facts like in English "men chuckle, women giggle" has been used as examples of bias. That's not bias though, that's just how the English language works.
For example:
https://ai.googleblog.com/2020/04/a-scalable-approach-to-red...
I wish this were true.
>Timnit responded with an email requiring that a number of conditions be met in order for her to continue working at Google, including revealing the identities of every person who Megan and I had spoken to and consulted as part of the review of the paper and the exact feedback. Timnit wrote that if we didn’t meet these demands, she would leave Google and work on an end date. We accept and respect her decision to resign from Google.
Self entitlement or she forgot she was just an employee, and not some sort of independent tribal leader fighting google from the inside.
It's not clear to me here what you're advocating for -- a lawsuit wouldn't be raised by such an intelligent and virtuous individual unless there was some clear wrongdoing. Are you saying that you think workers shouldn't sue their employers if their employers are breaking the law?
In any case, Timnit Gebru (this particular AI researcher) has been recognized as having done some pretty important work in AI ethics. Sounds like this whole thing is messy, but thought I'd mention that for more context.
I'm sure LeCun knows this too. It's a fact that has been known for almost 40 years now, that for a given generalization system multiple bias modes are are possible, and of course the process of optimizing architectures against a biased benchmark will change the learning bias of the model. It's not just datasets that are biased, generalization systems are necessarily biased too.
As for the last line, I don't know that it should be expected for employers to fire employees pursuing recourse against sexual harassment.
I'm sure we can agree that the tone wasn't good, but she does have a point in saying that it's not possible to reduce the bias to datasets used to train the final model.
The paper : http://www.cs.cmu.edu/~tom/pubs/NeedForBias_1980.pdf
The Twitter replies read a lot like her Google message thing. She’s got a lot of good things to say but needs to work on the delivery.
I think she’s playing the victim and race card a lot more than she should. She’s got all the ammo she needs to make her arguments without the angry victim tone.
Based on a long career of watching people trying to convince the others, attacking people is typically a failing strategy. Instead, learn how to message in a way that is non confrontational, and most importantly, make it clear what is your opinion, versus fact.
When she speaks people listen. Nobody listens to wil421.
>I’m always amazed at how people can continue to do thing after thing like this and then turn around and ask me for some sort of extra DEI work or input. This happened to me last year. I was in the middle of a potential lawsuit for which Kat Herller and I hired feminist lawyers who threatened to sue Google (which is when they backed off--before that Google lawyers were prepared to throw us under the bus and our leaders were following as instructed) and the next day I get some random “impact award.” Pure gaslighting.
I think if you threaten to sue your employer, the employer isn't too much at fault and just covering their own neck when you later threaten to resign and they take you up on that offer.