Agreed, it bothers me that this tactic is so successful. I could not find an instance in this "discussion" where Gebru outlined any concrete steps to solve for, or lessen bias, in ML. It seems she mainly wants to complain, rather than fix the problem. I have been googling about ideas to lessen ML bias, and the best I've found is "diverse training datasets" which Gebru herself says "is not enough".

Well what is then? Outrage without a solution is useless.

The discussion mentioned the Tutorial on Fairness Accountability Transparency and Ethics in Computer Vision at CVPR 2020. The tutorial was recorded in three parts. The third part[1] includes solutions. These are either direct quotes or paraphrases of the bullet points contained therein.

1. Disaggregated Evaluations[2][3], Counterfactual Testing[4], Interpretability Methods[5]

2. Recognize [l]imitations(sic) of technical approaches (this discussion)

3. Model documentation frameworks[2]

4. Standardized framework for transparent dataset documentation [6]

5. Positionality awareness [7]

6. Actively follow the perspectives of people in marginalized groups

7. Make intentional design choices to privilege the perspectives of marginalized stakeholders who are at most risk of being harmed by the technology we develop

8. Value interdisciplinary and 'non-technical' work

Note: These are some of the sources given as examples. Some are omitted for the sake of time.

Also note: Just because there are points doesn't obligate you to agree with them

[1]. https://www.youtube.com/watch?v=vpPpwa7W93I&t=1499s

[2] https://arxiv.org/pdf/1810.03993.pdf

[3] http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a...

[4] https://research.google/pubs/pub46743/

[5] https://arxiv.org/pdf/1711.11443.pdf

[6] https://arxiv.org/pdf/1803.09010.pdf

[7] https://dl.acm.org/doi/abs/10.1145/3351095.3375666

To clarify, The Tutorial on Fairness Accountability Transparency and Ethics in Computer Vision is not a technical presentation, it is very much a intersectionalistic take on the topic, more concerned with power, oppression, and social justice than engineering.

Suggested technical solutions [1 above] was exactly one sentence and the only sentence in the presentation that suggested any sort of technical approach, presented without elaboration, and the recommended solutions like Counterfactual Testing are also methods for fixing the training data.

The remaining 99% of the presentation was spent on the idea that if AI engineers and scientists were more diverse and attended more diversity training, the models they produce hopefully wouldn't be biased.

I think this is a fundamental misunderstanding.

Suppose I dismissed cryptanalysis by suggesting anyone involved in breaking cryptography ought to produce better themselves.

Suppose I dismissed a piece of medical research that shows some drug is no better than a placebo, and I demanded that those researchers produce their own drug that works better than the one they presume to criticise?

Critique is valuable. The entire field of science itself is built on absorbing critique, and checks and balances to ensure we reason out of robustness rather than hasty assertions.

In other words: suppose Gebru identifies that some particular ML model is racially biased. That insight, if true, is in itself a valuable contribution to the field. Suppose Gebru further develops an argument that the nature of this bias is (or is not) one in which different choices in training data will not substantially solve. That, too, is itself a valuable contribution to the field, and this kind of work is not at all the same thing as pointless "outrage".

I think the Yann's main point is that this specific model's racial bias was a training data problem; Gebru has not provided any evidence otherwise, as far as I know, apart from repeatedly asserting that better training data won't fix the problem.

It would indeed be a valuable contribution if Gebru could have shown how better training data would not have fixed the problem (e.g. by feeding better training data to the same model and still reproducing the issue).

But as it is, she is asserting that better training data won't help without proof and not recommending any alternatives, while taking a hostile tone of conversation. I would be hard pressed to take her claims as good faith criticism.

Adding some more sources to show that these scholars are not raising issues without concrete solutions -- Gebru and others have proposed many solutions and more concrete problems than this easily solved question of statistical bias. They are mostly not focused on the issue of statistical bias, which is more or less beside the point to everyone, as LeCun and Gebru both pointed out.

Podcast:

- https://www.wnycstudios.org/podcasts/science-friday/segments...

Articles:

- https://www.propublica.org/article/machine-bias-risk-assessm...

- https://themarkup.org/locked-out/2020/05/28/access-denied-fa...

- https://ainowinstitute.org/AI_Now_2019_Report.pdf

Book:

- https://www.ruhabenjamin.com/race-after-technology

I need to double check but from what I can recall, while she didn't mention any solutions or alternatives, she did point him to her previous work about the topic. (but she wasn't specific and I don't think she even included a link)

If you have diverse datasets, but your [prison recidivism, credit risk, or whatever] model doesn't like [giving light sentences to, issuing credit to] things correlated with being black... what then?

Or, for facial recognition, etc-- if you have done all the research with white faces, and then someone complains the system doesn't work for black people-- what then? Just saying "oh, with a different training dataset" to handwave things away doesn't mean that the resultant system will work well for minorities. The camera/optical system/etc may not like black people, too, and without additional validation you can't find these effects.

IMO, you don't need to know how to fix a problem where injustice is occurring to complain about and protest the injustice.

To expand a bit on your question directly and agree with your other points obliquely

> If you have diverse datasets, but your ... model doesn't like ... things correlated with being black... what then?

How do you release your data/models today? Does your release process include a section detailing what the data implies or the limitations of the mode? How would your boss handle being exposed to these findings? If your boss still wants to use the data set or deploy the model knowing the limitations, do you care enough to do something about it? Maybe you have a different set of answers to these questions. Does that make asking them any less important?

Hire more minorities in ML. One of the few industries where forced diversity is a good thing since your models will be better (wider range of opinions and sources).

See, it's not actually hard. Gebru is just correct in saying you are not listening.

The underlying (racially biased) superresolution algorithm was called PULSE and the code was released as an artifact of this paper:

Paper: https://arxiv.org/pdf/2003.03808v1.pdf

Here are the authors: Sachit Menon, Alexandru Damian, Shijia Hu, Nikhil Ravi, Cynthia Rudin

They conveniently captured an image of them all in one place for us here - https://cdn.telanganatoday.com/wp-content/uploads/2020/06/Au...

Pretty diverse team if you ask me. Sprinkling underrepresented (and therefore 'underpowered') representation into engineering teams will solve a lot of issues, but I think it has yet to be proven that it's going to help ML bias.

You give too much credit to the people who build those systems. Having more minorities in ML is a good thing, but I don't think it's gonna change anything regarding bias in ML systems.

ML, like most programming fields, has quite a few minorities - many organizations are majority-minority.

Yes, but that doesn't mean representative of all minorities. Very few Black people and Native Americans, for instance, even if Asian people are represented well.

This is immaterial. What about blind folks or quadriplegics or folks with chromosomal defects. Adding a black person to a team and expecting them to somehow fix anything beyond what they are trained at working on is unfair and unrealistic.

I mentioned before but this is the team that created the PULSE algorithm that made Obama look like a white guy from Arizona - https://cdn.telanganatoday.com/wp-content/uploads/2020/06/Au...

When the rationale is that having someone with an experience of being disadvantaged might be particularly inclined to help build a less biased system, one hardly refutes it by pointing out there are a lot of [not very disadvantaged] minorities on the team.

But one might earn some pedantry points with the argument.

What's the chance that a black engineer on an ML team doing work like this isn't also from the 'corporate(or educated) class'.

https://news.ycombinator.com/item?id=23697472

Black cops shoot black men all day long. I don't see why black ML folks are going to be completely immune to all of the pressures that have allowed the domain to get to its current state. While I'm sure many would accept the challenge, it's waaay too much to ask or expect of them.

Yes absolutely this issue needs diverse perspectives and the priorities they bring, but it will fail without objective standards to ensure that technology in the field reliably meets our expectations.

> Yes absolutely this issue needs diverse perspectives and the priorities they bring, but it will fail without objective standards to ensure that technology in the field reliably meets our expectations.

It sounds like we mostly agree. It certainly doesn't hurt to have someone who's been the subject of unfair profiling before to get his spidey sense going at the implications of the system, but it isn't a cure-all.

Yep!

Sure, and in some cases that does matter. But it seems unlikely that the PULSE model was biased towards white people because of latent racism, given that 3 of the 5 researchers who built it weren't white.

I wanted to write to address your point about the technical assertion, without getting into your request that Gebru empathize with one of the most prominent figures in science.

LeCun focused on the narrow technical question of statistical bias. Gebru and many many others have raised the point that an algorithm which works equally well for all skin tones will still have a societal impact, and they want to question that impact. How does a given new technology shift power? In whose favor, and at whose expense? That's the question which is arguably more important than some accuracy on a benchmark -- which as LeCun points out is easily solved with a more representative dataset.

Have you engaged with any of the critical study of AI, from Gebru or others, like the AI Now Institute at NYU or Ruha Benjamin at Princeton, or one of the lots of other scholars even in that one Twitter thread? The results of their work have included the recent IBM and Amazon moratoria on the sale of computer vision technology, and likely others to come.

The fact that LeCun refused to engage with a valid and deep criticism of a paper (which was not even his), to me at least, indicates a lack of willingness to engage critically with the social impacts of computer vision. Gebru's tweet pointed out that lack of willingness and her frustration with it, and also the fact that LeCun has had ample time to hear of it (it is in the news a lot, especially after these Amazon and IBM announcements, but groups like AI Now have been describing this for years.)

How about some empathy for Gebru? I would argue that she actually has more on the line than the single most celebrated individual in computer vision in this situation.

But that's the point, Lecun didn't reject the social impact, he was just not interested in that discussion. He was interested in a scientific discussion on the main cause of the bias. That was the all purpose of his first tweet. Then her reaction was about listening to marginalized people, which had nothing to do with the discussion - he never denied that those systems have bias and social impact.

>> But that's the point, Lecun didn't reject the social impact, he was just not interested in that discussion. He was interested in a scientific discussion on the main cause of the bias.

This isn't allowed anymore, for better or worse. If you don't talk about things with a social justice lens on Twitter, you're liable to get called out as ignorant. It's the #1 subject on the site right now, and no place for discussing anything remotely adjacent to it without invoking it.

Aka “Silence is violence”.

Well yeah that's somewhat of a separate thing. If you say something but don't mention the right stuff, you get yelled at. If you are silent, you get yelled at.

In fairness, if you're gonna post a statement, you better go full bore with it with all the keywords in it. People were circulating rubrics on Twitter to grade social justice statements and brigading those who didn't hit all the notes.

https://twitter.com/peterhassett/status/1267452984432173058

He has the privilege to ignore that discussion. That doesn't mean everyone else does, or that other people can't be frustrated with that choice.

Wait...ignoring a discussion on Twitter is a privilege now? Some people don't have the privilege to do so? Just...put down your phone? I guess that makes me super privileged, seeing as I don't visit that cess pool of a site.

I'm not talking about ignoring the Twitter thread itself, but rather ignoring the issue it discusses. What I was getting at when I used the word "privilege" is that LeCun is not a member of a group which tends to experience negative effects from these kinds of technological tools -- quite the opposite. If you're a member of one of those marginalized groups, as Gebru is, those issues might become a little more interesting to you. For instance, these kinds of issues, https://www.propublica.org/article/machine-bias-risk-assessm..., https://themarkup.org/locked-out/2020/05/28/access-denied-fa..., https://www.ruhabenjamin.com/race-after-technology

Well, can you help me understand how the conversation was detoured in the first place? I tried to read it, but as mentioned I don't use Twitter so it's confusing to me.

If I read it right...a person posted a photo that turned pixelated Obama, a half white person, into a fully white person.

A person, who I assume is some ML expert, explained that it's because the training set used uses too many white faces.

A person replied very negatively to this answer, saying she's tired of it, that he doesn't listen, and talking about injustices in the world.

I don't think either of them are wrong, I just don't understand the leap. Turning pixelated Obama into some random white guy isn't a social injustice. I kinda felt like both weren't even participating in the same argument.

I think it does mean that other people can't be frustrated with that choice. It's very toxic behavior for me to jump into a conversation and tell people they're bad for not discussing other topics I find more important.

To be fair, LeCun started a discussion around bias in ML -- Gebru's reply is on topic, since statistical bias in ML is a narrow way of understanding the social impact of an algorithm. Gebru is one of the foremost scholars on that issue.

I can certainly empathize with Gebru. I also feel frustrated when people post bad takes on Twitter, especially when I know they know better.

That doesn't mean we should condone her decision to start yelling about it. Shouting matches are a bad thing, even when they take place on Twitter instead of a hallway - most people don't want to work in fields where disputes are resolved through shaming and caps lock.

If the problem is that the system is too efficient or too good (people need slack in their tethers —only catch 75% of tax cheats, deadbeat parents, etc.), I’m sure they can tweak the system to be “as efficient or accurate as people” but cheaper.

That's not the problem. To get a sense of it, let me punt to people who know more than me.

- https://www.propublica.org/article/machine-bias-risk-assessm...

- https://themarkup.org/locked-out/2020/05/28/access-denied-fa...

- https://ainowinstitute.org/AI_Now_2019_Report.pdf

I can recommend more if you get through those.

"The social impacts of computer vision" says more about the hype around the subject matter than anything. It's marketing, not science. Computer vision isn't that cool, yet. Running a deep convolutional network is a sales pitch by AWS to sell GPUs by the second.

I actually sell computer vision software - specifically designed to help reduce subjective evaluation of aesthetic material performance. Most computer vision is real prosaic. It's not scifi. You read a bunch of papers about horrible things done to marmot eyeballs in the 60s, build a discrete model because neural networks are notoriously hard to support in real-world applications where you have to explain yourself to a chemist, and move on. Her work is justifiably marginal.

I also work in computer vision (for biological sciences). I use deep convnets every day, and certainly they are increasingly central in biology (https://arxiv.org/abs/1611.00421), proteomics (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3837439/, https://deepmind.com/blog/article/AlphaFold-Using-AI-for-sci...), astronomy (https://arxiv.org/abs/1902.05965), chemistry (https://www.nature.com/articles/s42254-019-0086-7, https://pubs.rsc.org/en/content/articlelanding/2019/sc/c8sc0...), many other sciences. I think it's great, they're enabling progress in lots of really tough scientific problems. And for what it's worth, people are still doing horrible things to mice, salamanders, zebrafish, haha, being a model organism is not a good gig, but of course it's still a huge part of neuroscience...

Personally, I don't think that an interest in the social impacts of one's work as a researcher should be considered hype -- shouldn't one be interested in their impact on the world? For computer vision in particular, these systems are widely deployed. Also, they are just part of a larger apparatus of automatic decision systems that everyone now interacts with every single day, from getting your insurance rates to getting a loan to... Those have now become prosaic too -- certainly not scifi.

He didn’t quit his day job. He just quit a social media app. He will survive.

just shows, some people are more interested in the fight than working to actually address the issue

Indeed! She was the bully in that all debate. I also thought about narcissism when reading her tweets. It looked like tweets that someone like Trump could write.

He's right. Twitter is pretty toxic. I'm not sure if FB is better for such conversations and debates. But he's active there in case you want to continue following him.