As a matter of employment law, it's generally pretty difficult to claw back pay from an employee. And this is almost always a good thing.

I'd assume that such a policy change could actually have severe unintended consequences as it would necessitate some metric to evaluate this, which it is notoriously difficult to do with respect to research. Not to mention that most research that is wrong is not wrong to intentional acts of deception, but just that people make mistakes. I'd assume the consequences would result in this territory, making mistakes far more costly. Not something we want for research, where essentially we're trying to learn how to play the rules to a game by playing the game. Often mistakes help us learn those rules.

No, universities are a reputation economy and do their best to avoid reacting to or calling attention to fraud. They benefit from the perception that their professors are all competent and honest. This particular incident has got some press pickup due to the irony of an ethics professor who wrote a book about breaking the rules at work being caught cheating, but academic fraud is common and people who report it say that almost invariably nothing ever happens. Even in the rare cases when there's an investigation, it all gets swept under the rug. Note that:

• Harvard took over a year to request retractions, even after outsiders did the heavy lifting for them.

• This prof has silently been put on leave.

• Harvard won't comment.

• She has many coauthors on the bad papers. Nothing has happened to them. They were all blissfully ignorant.

> most research that is wrong is not wrong to intentional acts of deception, but just that people make mistakes

In reality the line is blurry to the point of being meaningless. Many glaring problems in research papers get claimed to be "mistakes" even when there's no plausible way they could be so. You have to be willing to penalize mistakes too, otherwise you can never penalize fraud.

As an example, check out the story at the top of this essay I wrote a couple of years ago:

https://blog.plan99.net/fake-science-part-i-7e9764571422

The before-surgery and after-surgery photos are clearly pixelwise identical except for the skin lesions, which were airbrushed out. Despite the sheer visual obviousness of this fraud the journal didn't notice, and when outsiders did the authors simply claimed “The photograph was taken in the same room with a similar environment; unfortunately the patient wore the same shirt” and the journal accepted this explanation! A ten year old could do better than that.

BTW don't conflate the unknown nature of answers to research questions, with the known nature of how to scientifically answer questions. The rules aren't hard, especially when you start from a base as low as "don't fake your data".

As to your other comments and blog, I'm well aware. And if I may add some constructive criticism, if you want to call something pixelwise identical, it helps to show a differencing map. It just makes the claim substantially stronger and clearer, especially since there's a lot of additional factors that can change pixels, making the likelihood of a match even lower. These are things that wouldn't be possible by their explanation, whereas human interpretation is more subjective (it'll be a bit messier than this though). Though even low differences are still pretty damning.

> with the known nature of how to scientifically answer questions.

I think this is a bit naive though. Generally in science we rule things out, not rule in. That's why I'd be hard pressed to say that we have a "known nature of how to scientifically answer questions" because "scientifically" is doing a lot of heavy lifting and is fuzzy. Essentially "there's wrong ways to do science, but not a clear 'right way.'" Obviously faking data isn't okay nor is just doing something and calling it an experiment. Rigor is necessary but what constitutes sufficient rigor is dependent on the strength of the claims and thus the uncertainty of those claims. A big issue we face is that we take empirical results as equivalent to analytically derived theoretical models combined with experimental results. i.e. Physics develops a theoretical mathematical model and tests experiments against that whereas {psych,medicine,ML} make a non-mathematical hypothesis and report results. The problem with the latter is that it is harder to capture uncertainty because it isn't as strictly bounded. But it is still science, and the reason this is done is because the models are likely intractable (if not provably so). But that's why Asimov wrote about the Relativity of Wrong, rather than the precise way in which we can exactly do science. In fact, if the latter was true, we'd program machines to do it. Just a bit more nuanced.

Point well taken. The a-theoretical nature of the social sciences (and many others) has come up before. "A problem in theory" discusses this:

https://www.nature.com/articles/s41562-018-0522-1

As for science, my main point is that we need to embrace the noise. A lot of people try to use metrics to reduce the noise (which is good) but have this fundamental belief that noise can be absolutely removed. But this is a deep misunderstanding of what noise is. Ignoring the probabilistic nature of the universe, even a fully deterministic system has noise. A good example is the lack of absolute precision. Statistics captures error and helps us analyze that error. But error always exists due to the lack of both omniscience and omnipotence. In an nondeterministic world, it just becomes more important to embrace it. To acknowledge that it is a fundamental part of not just the model, but the thing we're modeling. In this respect, removing a noise variable doesn't just result in overconfident results, but missing a fundamental variable. Of course, noise can also be used to capture unknowns. So in any case, the noise has the be embraced. We should reduce it as much as possible, but it can't ever be removed. That's the problem: people are removing noise from their models (e.g. using a benchmark to determine evaluation absolutely)

What policies are those? I thought 'publish or perish' was an expected outcome given the extreme supply/demand situation in the academic job market.

I don't think anyone would argue otherwise. The issue is that if there's a culture that encourages (incentivizes, not advocates) cheating, then we should expect a culture where more people cheat. I think right now it is hard to argue that the current culture doesn't encourage cheating, and if we're being honest, doesn't have a good means of identifying it (publish or perish also decreases reviewing quality).

There is utility in both speed and in a slow step. The truth is that we need both. A continuum of methods and ideas. After all, we are searching in the dark and our only real advantage is parallel Monte Carlo search. The problem is that this is incredibly difficult to accurately evaluate and we are arrogantly attempting to evaluate exceptionally noisy systems with extreme precision without even having a basic model of that system. We could relate this to businesses too, but the timetables for them are typically shorter and so the chaff can more clearly be gained (often post-explained through pure intent instead of it's combination with luck). Then again, we are trying to make academia a business, trying to apply their noisy successes to a completely different model.

Ugh.

There's plenty of honest researchers in the field.

You don't have to be corrupt to build a nice career and, now that I think of it, only people who are equally corrupt seem to embrace/justify/tolerate these practices.

This is multiplied several-fold when you are doing this populist intellectual dinner talk circuit in one of several countries whose elite are anxious to establish (or regain) their own scientific prestige on the world academic stage and will throw unbelievable amounts of money at the most brittle and flimsy proposals so long as there is an R1 institution attached.

Exactly. We are finally starting to see the unwinding of the 20th century scientist myth. With the advent of corporate culture in the academia, it now attracts and nurtures very different personality types.