Doing bad science is an issue that all scientific fields need to be vigilant about (e.g. the "replication crisis" in psychology). It is particularly relevant to machine learning at this point in time as i) the field is growing extremely rapidly and ii) there is lots of $s on the table for those who achieve commercialisable results. So the incentives to do bad science are increasing while the checks against are weakening. These forces are not at play in, say, theoretical CS.

> e.g. the "replication crisis" in psychology

Just an aside, it's not just psychology. When you google "replication crisis" most articles say "... in science" for good reason.

https://en.wikipedia.org/wiki/Replication_crisis

When mentioning "replication crisis" I should probably add articles like this one too: https://www.insidehighered.com/news/2018/04/05/scholar-chall...

That article is incredibly frightening.

> to conclude that, although misconduct and questionable research methods do occur in “relatively small” frequencies,

which is kind of expected in a career track that doesn't exactly come with hedge fund-type money -- why would many people be dishonest for grants?

> there is “no evidence” that the issue is growing.

which might mean that the issue isn't in how science is practiced nowadays, but that the scientific process (grants, publication, peer review, research programs) has always been broken.

It's not clear that any of these is the case, but it's depressing to see "well, people are generally not crooks and the game has always been played this way" as an optimistic view of the replication crisis.

While outright fraud is probably rare, "questionable research methods/practices" are standard. Most people doing them dont even know there is something wrong with it. Pick any biomed or social science paper and I will quickly find some indication of them (lack of blinding, weird unexplained sample size changes, missing controls, failure to show a direct comparison of the variables you claim are related, etc).