Edit: And to be clear, her most cited paper (> 1000 citations) had an impact on what some companies did: https://www.theverge.com/2020/6/8/21284683/ibm-no-longer-gen...
What is disturbing here, from a researcher's point of view (even within a company!), is the lack of transparency on why a paper is being blocked from publication. The implicit understanding of being a researcher in a company like Google is that, while papers are subject to review prior to submission/publication, that the company will not wholesale reject it and instead will work with you to make modifications and changes. Clearly something different happened here that should be of concern to people considering working at Google Brain.
Does the same paper get 1000+ citations if it were published in 2010? Or even 2015? Probably not.
Also, anyone who works in the field of AI fairness and ethics research knows about it. So yeah, it's an important study.
What does that tell you?
As far as I can see, the only way she would keep her job, is if her research was so fundamental to the company that they literally could not function without her. Citations don’t tell you that. My boss is a star researcher in his field with multiple 1000+ citation papers. He still has to act respectfully of his boss and peers.
Vs
FAANG hires top diverse ethics researchers for good publicity and they demand clout/unreasonable changes that aren't in FAANG's interest and they get kicked resulting in bad publicity.
Unclear which is happening here. I agree it isn't in FAANG interest to hire these people but that misses the point.
Industry research is not academia, where you'd expect to be free to choose the avenues of your research, and you should expect that publication is restricted. The technical improvements you make may be kept under wraps to gain a competitive advantage; and policy agenda work is essentally part of PR strategy, it's accepted if and only if that agenda benefits the strategic interests of the company. Tenure in academia offers some freedom of research (though not even then abolute), but when working for private companies, you don't have that freedom unless you have a specific contract asserting that, and even then you'd be wary of biting the hand that feeds you.
Like, if you want to expose climate change risks, it's not going to work out while employed in an oil company, and if you want to advocate for regulation of ML research while working in a company that would be bound by this regulation, this is certainly something that the management would be expected to care about.
I'm not sure Google Brain's recruiting pitch includes a description of them blocking publication of a paper you'd worked a long time on (and even given them a heads up about, as Gebru apparently had). Their pitch is almost certainly about being able to do research similar to how you would in academia (except with more money and resources, etc.).
Companies hire activists because they want to look good without actually changing things. Activists go to work for companies because they want to change things. So, naturally, the company has to lie to the activist about their function to do this, and the activist has to be naive about it. If the activist starts doing what they want to do, and not what the company wants them to do, then the company will fire them, and the activist will reach for an explanation.
"Look how awesome this activist is we're hiring," easily becomes "Look how unreasonable this employee is we're firing."
I agree with this statement but for the life of me I can’t come up with a satisfactory answer as to why companies don’t want to actually change things. Is it purely institutional inertia? Are the benefits of a more diverse workplace not accepted by the powers that be, so efforts to achieve that are undermined?
The company doesn't "want" anything. It's a big social machine that is not so much designed as evolved, and only the very core of the company experiences selection pressure.
At the center of every company is a system that prints money. This bit is almost all that really matters, and is the part that experiences selection pressure.
Threaten it, screw with it, or break it and you will be out of a job, because you either killed the company or got thrown out by people who were afraid you would (or, sometimes, who are just greedy and were worried you'd hurt the amount of money they make through the company).
Everything outside of that central system is secondary, and doesn't really impact the company's survival a hoot.
People at the company may love their secondary systems, really believe they matter, and pour sincere effort and careers into them, but when secondary systems come into conflict with primary systems, they Lose. Every. Time.
In the very long term, nothing else can happen, at least in for-profit companies. Competition will destroy kinder companies by being ruthless to secondary systems that the kinder ones keep. This is what Scott Alexander of Slate Star Codex calls the "Malthusian race to the bottom". Amazon and Wal-Mart are locked in a violent struggle of this nature right now.
I think most people are not entirely aware of these dynamics, as my premise that companies are mostly evolved not designed suggests.
So, in all likelihood, there are very sincere people who really want to see the company change for the better recruiting activists and noble researchers.
Everyone likes to feel that they're being goodness, so they all applaud the secondary "goodness optimization" systems.
After enough time, though, those activists come into conflict with a primary system, and discover to their shock that they don't have the leverage to change it at all. This is particularly astonishing to them because they've fought to change a lot of secondary systems and had many successes there.
They (rightly, from their frame of reference) refuse to budge, get fired or leave, wash their hands of the place, then start at the next place in one of its secondary systems.
Rinse and repeat.
It's hard being human.
If Timnit Gebru were a cynical opportunist, she would just hop on the bandwagon, pretend to care and collect her paycheck. But she seems to be a true believer, and true believers won't accept ineffectual groveling. They want to see the real thing. Ergo, collision.
In this case, both Google (a social justice activist gets them into a public screaming match, what a surprise) and Ms. Gebru (played chicken with her employer and lost).
"As far as I can tell" is doing a lot of work here.
I can confirm this is untrue. I worked with someone like Gebru (definitely an activist researcher) at a non-FAANG and AI ethics drove 3 multimillion dollar deals. Was a better business case than most I built over that time.
Besides, saying they don't drive revenue is a critique of research roles, not Gebru...
Also, let's be real. How much revenue are the vast majority of AI/ML researchers actually driving in these companies? Sure, someone gets promoted for multiple NeurIPS papers, and it looks great for the company. But most of the time, it doesn't directly impact products and revenue -- in the short-term -- and that's ok. Same deal for AI ethics and policy researchers.
It has become an archetype at this point.
But that said I can see some similarities with the whole politics at work landmine issue.
Judging by your interpretation of James memo I'm leaning towards that direction. Especially considering the fact James made multiple direct claims in the memo he was not suggesting anything of the sort which you are alleging (I have a feeling few people actually read the memo).
Regardless the two stories were very very different. Nor does his story match my comment's 'archetype' of a certain type of employee who always end up flaming out of their job, usually via making non-conditional demands (unsurprisingly coming from a bubble/worldview where it's entirely normal to have things which are not open to discussion).
Even more than that, it's far easier in infosec to say "There is gatekeeping going on. I'm going off to make my own space where you're not invited."
The idea that companies only need to do maximize shareholder value isn't that old and was only popularized in the 80s. If you restrict creating value to that, then maybe she didn't create any at Google. But maybe Google wants to create a software that works equally well for everyone and raising the issue of IA biases would have helped achieve that goal.
How is this either/or? Highly educated people can still join fringe political clubs.
There is nothing about technical education that immunizes you against radical politics. At the risk of Godwinizing the reply: the engineers who built the first liquid fuelled rockets in Peenemunde were N..z..s, including the guy who was later brought into the US and built all kind of powerful and ingenious rockets there (Wernher von Braun).
Maybe if you’re going to hire ethics researchers, you should listen to them?
Context:
> Timnit co-authored a paper with four fellow Googlers as well as some external collaborators that needed to go through our review process (as is the case with all externally submitted papers). We’ve approved dozens of papers that Timnit and/or the other Googlers have authored and then published, but as you know, papers often require changes during the internal review process (or are even deemed unsuitable for submission).
And the key point:
> Unfortunately, this particular paper was only shared with a day’s notice before its deadline — we require two weeks for this sort of review — and then instead of awaiting reviewer feedback, it was approved for submission and submitted.
Not sure who approved for submission. Gebru herself or by someone in management? Then they address the weaknesses of the paper (as part of the review process).
> A cross functional team then reviewed the paper as part of our regular process and the authors were informed that it didn’t meet our bar for publication and were given feedback about why. It ignored too much relevant research — for example, it talked about the environmental impact of large models, but disregarded subsequent research showing much greater efficiencies. Similarly, it raised concerns about bias in language models, but didn’t take into account recent research to mitigate these issues. We acknowledge that the authors were extremely disappointed with the decision that Megan and I ultimately made, especially as they’d already submitted the paper.
Granted, its hard to track the timelines and who said what at what time, but there seems to be reasons why the paper would of failed their internal review process.
Doing work that needs to be actively suppressed is not producing value - at least not to the company. So apparently there's a fundamental misalignment between the work Dr. Gebru wants to do and the work that Google wants her to do; this was made clear with the suppression of the paper, and escalated as Dr. Gebru apparently (and understandably) did not want to work in a very different direction, (quoting her tweet) "asked for simple conditions first" which were totally unacceptable to the employer ("We cannot agree to #1 and #2 as you are requesting."), so that's it, if neither is willing to compromise on their primary goals, they can't work together.
This does indicate that apparently listening to ethics researchers they hired is unacceptable to Google. Sad, but not surprising.
I don't know about publishing standards at Google but, assuming they are uniform for all researchers, I am pretty sure Google would be unable to retain talent if researchers were asked to prove value to Google for every paper they publish. I think for researchers, this reinforces the oft-dismissed notion that you give up significant academic freedom in exchange for a fat paycheck when you join an industrial research lab.
This can be particularly damaging for Google because many of its research projects don't have an immediate business impact, and it hires top researchers by offering an alternative to the slog of academia.
IMO it’s the _companies_ that strike me as toxic drama queens with their shady tactics and obfuscations, hiring ethics and policy folks to look good without any intent on following through with support or respect.
Just to be clear, their end of the deal is to justify Google's actions as ethical, not to ensure that Google's actions are ethical.
> I think this is all a good thing because it may alert some executives to the price for hiring activists into research roles.
Ah, yes. They're going to do their jobs and point out poor ethics and policy in _your_ company. I mean, they were hired for that, but if they could just look down and _pretend_ to care about the space they were hired for, instead of actually caring, that'd be great.
From what I can tell, she doesn't seem to address the points raised at all and instead complains they won't tell her exactly who said what.
https://www.nature.com/articles/nn0399_197
I think the process being opaque is more detrimental, because there's no way for anyone outside of the process to know it happens. If it wasn't opaque, then everyone could see when it happens.
Not being able to engage in a debate about the merits of something is also sketchy. It seems like an order being dressed-up as peer feedback.
I mean, from the tweets what I gather is she wasn't planning on leaving just yet. Saying "hey, you want to resign by [X] but we'd rather you be gone so let us inform you've just been resigned as of now" is essentially firing [1].
> Offering to resign was a huge mistake.
I think this will be good for her in the long term. I'm sure she'll find another research position, and the alternative was betraying her convictions or just the huge burden of constantly being worn down by Google.
[1]: https://twitter.com/timnitGebru/status/1334364735446331392
[1] https://twitter.com/timnitGebru/status/1334353073012158464
They may be wrong, but it's a valid coinage that has worryingly become mainstream enough to be both a cliche and indicative of something deeply wrong with our society.
> You are not worth having any conversations about this, since you are not someone whose humanity (let alone expertise recognized by journalists, governments, scientists, civic organizations such as the electronic frontiers foundation etc) is acknowledged or valued in this company.
It feels more generally accurate to say that no ones humanity is respected at companies like Google. That isn't to discount or lessen the difficulties minorities and women have in tech companies; those are substantial and significant. But if someone in a diversity role is slighted, the natural place to jump is a diversity motive; its far more likely that the company just doesn't give a shit about anyone.
And that's true of nearly every company. It was immature for us, as a community, to believe big tech would be any different than any other megacorp. They say they care about diversity, inclusion, and people; they don't. They don't care about any individual. Companies, as a collective intelligence, care about two things: Money and Self-Preservation (in that order).
That being said, I totally support the fight that people like Timnit are fighting; they're fighting for the recognition of humanity, through the lens of diversity and inclusion, a critical component of a much larger fight intrinsic to a capitalistic society. That's the value of a hire like this; its just the unimaginably unfortunate case that this value hasn't been realized at Google, and probably never will.
>Said hires have thus far, as far as I can tell, not produced anything of substantial value to any of these companies.
These two statements are at odds with each other.
I mean, yes. It's an unfortunate reality that ethics are often at direct odds with the unrelenting pursuit of business value. For a big tech like Google, by caring about people and the impact of your technologies, you will almost surely sacrifice profits in the short term.
So then the question is: people or profits?
Who determines what value a business gets assigned?
Seems to be a very broad brush that you're painting with. "Progressive people" encompasses a wide variety of individuals and beliefs.
Said hires have thus far, as far as I can tell, not produced anything of substantial value to any of these companies. They nevertheless become increasingly emboldened and misinterpret their situation. After all, no-one dares criticise them, they are showered with feel-good publicity. The only thing they need to do is keep their end of the deal.
Until they overstep, get fired, and cause drama by claiming everything that happened to them was due to discrimination, and not because they are toxic and disruptive.
I think this is all a good thing because it may alert some executives to the price for hiring activists into research roles.
Edit: Should add that I am purely talking about the power dynamics here, not research impact or the question of whether AI policy is an interesting subject. It is, I am just observing that it has not driven particular value to companies when compared to the negative publicity these dramas cause.