AI ethics are important and I'm willing to bet that the Google execs want to know about biases and other problems in their products and probably don't care that much what is published. They probably don't want high maintenance researchers that cause them headaches and are obsessed with microaggressions, etc. I expect that this stuff has been a distraction for too long and they are happy to be rid of this and happy for the signal it sends across the company.
I have little sympathy or concern for Ivy educated elite in-fighting. There is probably a huge number of qualified people just dying to have the opportunity to work there, have access to those resources and do research.
Tweet explaining how she "resigned": https://twitter.com/timnitGebru/status/1334343577044979712?s...
An external reviewer of the paper: https://www.reddit.com/r/MachineLearning/comments/k69eq0/n_t...
I think you should be more open about what potentially actually happened because your first sentence seems very accusatory without proper information, the very bias that Gebru speaks of. According to her email that Dean references in his, her "feedback" for the paper was given in the form of a confidential HR document. That seems highly nonstandard for a review process supposedly only concerned with scientific rigor and is not addressed at all by Dean's statement. Further, his statement clearly, likely intentionally, muddies the water about what the actual sequence of events were that led to the paper being created, internal reviewers originally being notified, when the submission happened, who approved it, who submitted it externally, who asked for this to be retracted, and when and in what order all this happened. The fact that his statement, that was apparently meant to clarify these exact proceedings, makes it even more vague about what actually happened seems pretty damning to Dean's case, in my opinion.
Both Gebru's and Dean's statements have very little overlap in flavor and in facts, so it's pretty apparent something is going on here that is nontrivial and abnormal.
The non-research motive is arguably to control the narrative about using ML (I admit I still choke at calling it AI) and big data techniques. I fail to see how this is advanced by having a very public spat.
I've no doubt some senior exec decided she had to go, but I don't think it is because of any intersectional reason, or to cover up any particular publication, but that she wasn't seen as an asset to the company. Strategies for negotiating exercise of power are very different at Google to social sciences academia (or twitter, which is engorged with righteousness over this), which she seems not to have grasped. There are few enfants terribles in corporate tech that don't have controlling stock.
Publishing a piece of work for the sake of publishing by ignoring the processes that are put in place is outright irresponsible and that's the end of it.
There's a classic documentary about this phenomenon: https://www.youtube.com/watch?v=9UiF7BsC1Ig
To be clear, both of these had been received for the paper in question.
> Unfortunately, this particular paper was only shared with a day’s notice before its deadline — we require two weeks for this sort of review — and then instead of awaiting reviewer feedback, it was approved for submission and submitted.
It seems like someone short-circuited the review process and submitted without review to meet a deadline, with a post-submission review. When this occurred, it was not all green lights. The expectation is that you then pull the submission and, post review, submit elsewhere. After all, conferences are all virtual now, it is not a hardship to submit elsewhere.
If you have other information as to the facts or the internal process do tell.
A summary of the events, as best as I can tell, is the normal presubmission review was done. After the paper was approved and submitted, someone, whether this was upper management or some other entity, did some additional review and required that the authors withdraw (?, but retract is the wrong term) the paper.
From what I know, both firsthand and from other sources[0], while not spotless, the issues with the paper were mostly nitpicks and fairly straightforward to resolve. That they weren't even provided until some escalation from Dr. Gebru is strange. That even after they were, sort of, provided, she and her team were not given the option to address them in the paper, is extraordinarily strange.
[0]: https://www.reddit.com/r/MachineLearning/comments/k69eq0/n_t...
What makes you think they weren't given the opportunity? We were told specifically they _were_ given an opportunity to revise, and instead, demanded HR(!?) provide attributed versions of any and all statements made by any colleagues regarding the paper (!??!!?), or they'd 'work on setting an end date'
I'm not exactly proud, excited, or anything but pensive about this series of developments, and it's _highly_ likely Jeff feels the same way. Everything got too hostile and out of control here for people to be able to work together healthily moving forward, and I'm skeptical of anything beyond that being anything more than people attaching a narrative to an unfortunate breakdown in a relationship.
His words were, and I quote "we require two weeks for this sort of review". That is an absolutely false statement, and Jeff knows it. I see no other characterization than a lie. I take no solace in that fact, priors were that Jeff Dean was an above average executive and ran a more ethical then average org. I'm disappointed to see my trust was misplaced and that Jeff was willing to endorse that document, but I'm not going to not hold someone accountable when I see them do something obviously wrong, even if we work for the same employer. Ethics extend beyond that.
> 1 day is certainly unreasonable.
It is not though. It's the norm.
> We were told specifically they _were_ given an opportunity to revise
This is not true (if you believe this to be true, can you cite said statement?). I'd be more than happy to discuss this with you privately, my username should be obvious ;)
> It is not though. It's the norm.
Not sure what I'm supposed to do with this. A literal intrepretation has you claiming that all Google researchers only allow 24 hours of review of their papers before submitting them for publication. That sounds wrong!
Fwiw, I'm not sure what you mean by legalistic. I see Jeff's interpretation as more legalistic: this is the policy, while mine is based on practice: the policy isn't enforced, and here's what people actually do. And like I said, it brings me no joy to see that Jeff is lying to justify this. I have a deep respect for his technical achievements, and prior to this my understanding was that he was an above average executive. Perhaps he still is. But that's not an excuse for me to not levy criticism at him when I see him do something wrong.
In practice, Google doesn't require two weeks for pub approvals. I'd be more than happy to provide the underlying date I'm basing that statement on, but obviously not here. Granted, if you want you can find the data for yourself and if encourage you to do so!
> . A literal intrepretation has you claiming that all Google researchers only allow 24 hours of review of their papers before submitting them for publication.
Given your experiences with Google reimbursement policy, why would you expect adherence to the written publication approval process to be different?
That's why I'd much prefer to have this conversation on corp. I can show you raw data and you can draw your own conclusions no need to listen to me or my interpretation whatsoever.
Full Disclosure: I work at Google.
https://www.ribbonfarm.com/2011/10/14/the-gervais-principle-...
> This is a simple and child-like example of the operation of a basic human instinct: the heads-I-win-tails-you-lose or HIWTYL (let’s pronounce that “hightail”) instinct. It is the tendency to grab more than your fair share of the rewards of success, and less than your fair share of the blame for failure.
Now whether one buys into that as literally true or not is one thing, but I think it is definitely a useful way of thinking about situations like these.
In fact, the “Golden Ticket Reconsidered” section at the link sounds somewhat like I imagined incentives within AI ethics research at Google are structured:
1. Cut a deal for performance-linked bonus for successful initiative
2. Set up a committee and charter it to collect, vet & recommend ideas
3. Drop hints & suggestions to create things that leadership favors
4. Create appropriate urgency in the work of the committee to achieve the risk-levels you want in the ideas produced.
The outcome of such a system:
> If it works, you praise everybody generously, hand out a few gift certificates, keep your bonus to yourself, and move on. If it fails, you blame the people in charge of the work for failing to consider an “obvious” (with 20/20 hindsight) issue. The chair of such a committee would likely be Clueless [a term of art in the blog series], his appointment being a false honor — a case of being set up take a fall
Further on, the author brings up the “Hanlon Dodge.” Dean’s characterization of Gebru’s termination (“resignation”) seems quite like that to me.
People claiming that the deficiencies in the paper are minor and wouldn't be blockers obviously have little experience submitting to academic journals. Other parts of Google doing deeply technical work probably don't have the same level of review as the Ethical AI group -- for obvious reasons.
There is usually a long back and forth -- there are even memes about the infuriating comments from "Reviewer 2" [1][2]. Omitting to mention argument-obsoleting developments in the field (from your own lab!) is more than enough to send you back to extensive redrafting.
To be clear on terminology -- a retraction is an academic black mark, and occurs to a paper after publication, usually for reasons of research misconduct. This is not an instance of that.
[1] https://twitter.com/redpenblackpen/status/113344056990719590... [2] http://jasonya.com/wp/wp-content/uploads/2016/01/PowerRespon...