There is a lot of "garbage" research that happens at universities. Yes, we do expect quite a bit of research to be dead-ends --that's the nature of research, but there is a lot of research that is knowingly fraudulent or uses very suspect data and or methodology.

Or is that standard unattainable because undue influence exists everywhere?

That isn’t to say everything is trash— lots of good stuff is still done with these requirements.

In a company you can always be fired and the singular goal is the bottom line. Much more so since modern companies seem to not operate with the mentality of the old corporate research labs like Bell, but keep a much tighter lid on things, always pushing for products.

Bias will always exist. The nature of that bias and the degree of impact will vary.

That would be the norm especially if your research paper is focused on new technology. For example you would write Herbet et co, in Journal Blah found results of X with method Y. However method Y has flaws of A, B, and C...

Even if a pervious study was utterly stupid in your opinion, if it's been widely publicized you still need to talk about it or else risk being accused of missing the basics.

The nature of the citation and how that impacts the paper doing the citing would need to be examined to form a meaningful opinion about Google’s behavior here.

Although let's be clear... this is bureaucratic issue. Some journals only publish English language papers. No matter how impactful your paper is, they'll never publish it if you wrote it in Tamil. Google only publishes papers that doesn't directly insult them, and spread fear about their own proprietary technology while providing no clear solutions. The paper may be brilliant but they definitely won't put their money and name behind it.

Since they (Google) are ideologically opposed to the paper's findings, I can't imagine they are citing it to talk about how brilliant it is though...

For example, a paper's intro might have an overview of how others have attempted to solve a problem before. Citations there will comprise a very concise summary of techniques from others, often pointing out limitations.

From the perspective of a PR team or agency, they aren't in a position to independently evaluate papers, let alone disavow a paper that carries the name of the company.

I should also mention that you can also pad your citations this way and build your h-index. It is metric hacking, but if it isn't too egregious this is normal. Classic example of Goodhart's law.

Imagine if Google had a standard that all papers they publish must be in Comic Sans font. Researcher refuses, publishes paper independently, and then someone at Google publishes a paper in Comic Sans referring to that paper. Is that wrong? After all, Google doesn't have a standard that all cited papers must be published in Comic Sans as well.

Google[0] claims the paper didn't meet the bar for publication and ignored too much relevant research. The authors obviously dispute these claims (I generally side with the authors that the "ignored research" line is a stretch).

The then amusing thing is that a number of Google authored or coauthored papers still cite it, for various reasons[1][2] (but there's another like half dozen or so Google papers at least in the ~500 citations). So the entire controversy is that the paper wasn't up to academic standards, except that empirically it clearly was.

> Were they reworked at all after being told they weren't up to standard?

There was an attempt by the authors to respond to the internal criticism, but this wasn't met constructively.

[0]: https://docs.google.com/document/d/1f2kYWDXwhzYnq8ebVtuk9CqQ... [1]: https://arxiv.org/pdf/2107.06499.pdf [2]: https://arxiv.org/pdf/2109.01652.pdf

Instead, maybe Timnit was fired/resignation accepted not because of her paper, but because 1) she was in management, and 2) she explicitly told her reports to stop the work they're doing, for "reasons", to the point where Jeff Dean had to come in and ask the employees to continue working on DEI programs: https://www.platformer.news/p/the-withering-email-that-got-a...

I didn't claim to know why exactly she was fired. I'm just claiming that Jeff's objection to her paper seems dubious.

And keep in mind her coauthor (Mitchell) was eventually fired. But yes, the question you ask is a good one, and I think the answer is revealing (the paper did in fact meet the bar for publication, and it was killed for political reasons, making Gebru's reactions quite reasonable!)

> she explicitly told her reports to stop the work they're doing,

No she didn't. She told them to refocus to be more effective. That's a completely reasonable! Quoting her email: "So if you would like to change things, I suggest focusing on leadership accountability and thinking through what types of pressures can also be applied from the outside."

That doesn't sound like "stop working on DEI" to me.

It's very clearly a call to stop focusing on ineffective things and to focus on more effective methods.

Really? Because people claiming to be the reviewers at the time said that the version they reviewed was a lot worse than the final version published much later. (And that version wasn't too great either.)

The internal special topics review process that wasn't formalized at the time but was pseudo-formalized later in response to this issue also supposedly allowed revision, but in practice this authors weren't given the opportunity to respond to the feedback, they were only give the option to remove their names from the paper.

So both your statement and mine can be true, as their were multiple reviewers in multiple review processes. To further disambiguate, the FAccT reviewers have noted that the paper improved during the academic peer review process. This is likely true. But that's wholly irrelevant to the line of discussion I was having, because that criticism wasn't "internal", and we can be sure it wasn't relevant because we know the internal reviewers have never identified themselves or made comment on the internal review process.

I recognize that this is complicated, and it doesn't help that afaict, Google has tried to blur the lines between their process and the normal academic peer review process.

In short, Jeff Dean (and others at Google) cited the finally published (not by Google) version of the Parrots paper. The citation [20] is in the context of the paper:

"The opening quote in Section 1 is based on a 2019 study from the University of Massachusetts (UMass) that estimated the environmental impact of training [2]. More than 1000 papers cite this paper as the source for the impact on carbon emissions of ML models, e.g., [1,6,7,17,18,19,20]."

in which the authors of this paper point out that the Parrots paper used what is likely a large overestimate of the carbon emissions from training. This is consistent with my understanding (that the power/emissions part of the Parrots paper isn't well backed by evidence).

Seriously, who thought that science for profit would net the best results for the collective benefit?

You take science but replace truth with profit as the ultimate objective. I'm sure that's going to work well.

The day we stop seeking profit in endeavors that can only be pursued properly at a complete economic loss, we're going to solve so many problems at once.

Nobody, and I don’t think “best results” or “collective benefit” were the goal when science started getting more commercialized.

The idea that good inventions would benefit all rather than patent holders must drive a true capitalist crazy.