News for hackers. Stuff that matters.
Re:Is Google’s AI research about to implode? by paulsutter on Tuesday February 23, 2021 @08:16AM (#26236886)
Of general interest, Jeff Dean’s comment on why the Gebru paper didn’t meet Google’s publication guidelines:

“Highlighting risks without pointing out methods for researchers and developers to understand and mitigate those risks misses the mark on helping with these problems”

from https://docs.google.com/document/d/1f2kYWDXwhzYnq8ebVtuk9CqQ...

Re:Is Google’s AI research about to implode? by dominotw on Tuesday February 23, 2021 @09:01AM (#26237283)
Isn't that asking too much. How would a researcher also know how mitigate the problems they identified.
Re:Is Google’s AI research about to implode? by refenestrator on Tuesday February 23, 2021 @09:48AM (#26237882)
The problems they identified have been known for years and there are lots of papers exploring how to mitigate.

The whole paper was a nothing burger wrapped in social justice language with asides about how global warming is Actually Racism because of disparate impact (interesting, not an ML topic).

If the problems aren't novel and you're proposing zero solutions, it shouldn't be a paper.

Re:Is Google’s AI research about to implode? by jl2718 on Tuesday February 23, 2021 @11:07AM (#26238902)
I wonder what would drive someone to do this. Anger? Loneliness? Work pressure? Imposter syndrome? Her prior work seemed more observational than theoretical, and the she got thrown into a top-level leadership position in a theoretical research organization with almost zero experience. Is there a single person that would have defended decision if that person was a straight able christian white or north asian male? (Note: please do not respond by speaking for others that do not share your own views.)

I wonder when the “performative wokeness” bubble will burst.

Re:Is Google’s AI research about to implode? by Traster on Tuesday February 23, 2021 @12:17PM (#26239883)
I don't know where you get these ridiculous ideas about imposter syndrome. Take one look at her career. She's written signal processing algorithms at Apple, won awards at conferences, got a PhD and worked for Microsoft in their AI ethics lab. She was more than qualified for her position and the research paper she was publishing passed peer review.

You're right, this wouldn't have happened if she were a straight white male - if she were a straight white male, there wouldn't be comments like yours making asinine assumptions about how good she was at her job.

Re:Is Google’s AI research about to implode? by refenestrator on Tuesday February 23, 2021 @12:27PM (#26240044)
An impressive career makes this paper's quality and the subsequent meltdown over it more puzzling, not less.
Re:Is Google’s AI research about to implode? by jl2718 on Tuesday February 23, 2021 @12:23PM (#26239983)
Non-sequitur. I invite you to disabuse me of any of my exact statements.
Re:Is Google’s AI research about to implode? by skinkestek on Tuesday February 23, 2021 @03:41PM (#26242593)
> You're right, this wouldn't have happened if she were a straight white male

If anything, straight white males are trashed harder around here.

That's the only problem I've ever had with having a woman on the team: the fear of saying something that offends someone, her or other.

Re:Is Google’s AI research about to implode? by throwaway19937 on Tuesday February 23, 2021 @12:56PM (#26240402)
This is #4 (Pollution of Agency) and #8 (Anomalousness) from Joanna Russ's How to Suppress Women's Writing (https://en.wikipedia.org/wiki/How_to_Suppress_Women%27s_Writ...).

Poorly written papers are a regular occurrence in any field - that doesn't justify attributing bad faith or character issues to Dr. Gebru.

Re:Is Google’s AI research about to implode? by ausbah on Tuesday February 23, 2021 @10:20AM (#26238256)
survey papers, reproducibility papers, and the like are still valid contributions to science
Re:Is Google’s AI research about to implode? by mlthoughts2018 on Tuesday February 23, 2021 @10:28AM (#26238365)
No, it is not too much to ask in the specific case of Gebru’s bad paper. Several of the arguments are specious, like comparing the total energy consumption for training GPT with car trips, or demanding that NLP researchers have to keep up with rapidly changing activist “woke” vocabulary and ensure their models are respecting it.

These are ridiculous claims, and it’s fair to respond to them by saying, “well, what exactly do you imagine a solution or mitigation looks like?”

Essentially, by the nature of how specious Gebru’s stated problems are, they demand clarity over what an “ethical solution” even is, conceptually, and why everyone would have to agree.

For example, you could discuss economies of scale or train-once-finetune-everywhere approaches with GPT that reduce total energy needs. Or you could discuss how researchers can register the corpus they use and the snapshot of time it was grabbed, with an open understanding that as long as the methods and data are reproducible, there is no research ethical issue with studying that corpus, no matter how much bias or lack of woke vocab a given person believes it has. (And also, nobody is required to just accept activist language as important or valid.)

Gebru did none of this. The article could literally be summed up by Gebru saying, “I think <supposedly shocking evidence> is bad, therefore its connection to something in ML is bad.”

E.g. “I think, subjectively, that the raw energy use to train GPT is bad. Here are some shocking comparisons. Therefore GPT is bad.”

It’s incredibly unrigorous and juvenile. Dean’s comments that it needs to clearly state mitigations is actually a super generous, polite way of saying the paper is just subjective amateur hour.

Re:Is Google’s AI research about to implode? by jgalt212 on Tuesday February 23, 2021 @09:55AM (#26237973)
Isn't that what the "directions for future research" section is for?
Re:Is Google’s AI research about to implode? by krallistic on Tuesday February 23, 2021 @10:27AM (#26238346)
Sure if no methods/work exists. In this case there already has been some work done, to mitigate these issues. So either mention them in related work and/or highlight why theses methods arent enough.
Re:Is Google’s AI research about to implode? by sp332 on Tuesday February 23, 2021 @09:31AM (#26237662)
Of course. Especially to expect all that to be in each individual published paper.
Re:Is Google’s AI research about to implode? by ausbah on Tuesday February 23, 2021 @10:18AM (#26238236)
wasn't the real issue that the standards they cited were quoted later in the process than normal