This doesn't always work but having the capacity to disbar lawyers for, say, embezzling trust accounts or remove a CPA's license and put them in prison for falsifying records, it certainly has a deterrent effect on similar crimes of trust.
How much worse would white collar crime be if everything was on the 'honor system' and there was no prison time for stealing from customers/clients/shareholders? That's what we have in academia: crime won, so now syndicates of crooks run the system. With other abuses of trust the material consequences are more readily apparent: someone's bank account is empty that should not be. With this abuse of trust, the damage is more to the integrity of the purportedly rational knowledge base. It's more of a 'reign of error' than a reign of terror.
It corrodes trust in our scientific institutions. When that pop something talk by an acclaimed psychologist turns out to be not only bullshit but outright fraud, it’s not a huge leap to project that mistrust onto e.g. the medical establishment or climate scientists.
Journals would then have to select studies based on their design and based on the question and scientists could just do science.
Edited to add (from Wikipedia):
> Over 200 journals offer a registered reports option (Centre for Open Science, 2019),[33] and the number of journals that are adopting registered reports is approximately doubling each year (Chambers et al., 2019).[34]
> Nature Human Behaviour has adopted the registered report format, as it “shift[s] the emphasis from the results of research to the questions that guide the research and the methods used to answer them”.[36]
> European Journal of Personality defines this format: “In a registered report, authors create a study proposal that includes theoretical and empirical background, research questions/hypotheses, and pilot data (if available). Upon submission, this proposal will then be reviewed prior to data collection, and if accepted, the paper resulting from this peer-reviewed procedure will be published, regardless of the study outcomes.”[37]
I think I agree that we need much more transparency and scientific rigor, first. Post the data whenever possible, incentivize people for creating good study protocols that make it harder for any single person to fake the data without it being obvious. Once we have enough confidence in that, let's do preregistration, sure. And then negative results should be as incentivized as positive ones, absolutely.
I worry about the order in which we want to do these things. If people are faking data, we need to make this specific step harder to hide from reviewers.
If you incentivize people to fail to replicate each other without solving the lag between fraudulent data and retractation, you will get fraudulent negative results.
That could cause a lot of confusion. If the replication crisis itself starts to be fraudulent, scientific credibility is further damaged, negative results will be less credible, and fraudsters will profit from the confusion.
Um sounds worse, (OT:) to add something a simpified (strict stripped) example -hope practicable:
You work at a company. Orders are comming in, wares are placed as 'reserved' till paid, than shipped. With a worker perspective, you check the emails with orders ('the customer whishes'), sending 'bills' and the wares if, or if not been paid - 'you've been paid'. Viewed by the company-manager your position may a) be valid oder come b) 'overvalued' (...)
Stucked in this example, you talked to other 'employeees' during break and 'there was a time meeting on weekends', so you talked about work and there was a (special) kind of 'more transparency' than without peer review. ^^
Regards, hope this helps... (-:
I'm only trying to make a point about the order in which we should shift incentives vs increase fraud-resistance requirements for experimental protocols.
I think the latter is more important, and should be done first.
The journal that publishes failed science would be the biggest and most important scientific journal IMO.
The only concern I have around a disincentive to not publishing is that in some fields, the peer review process can be quite flawed and effectively hold back worthwhile publications for various reasons - sometimes there's too much of an expectation of specific ingrained methodologies, or an expectation of certain "doctrines" to be adhered to.
There can also be some significant gaps between disciplines, where neither discipline wants to publish the work, as they both feel it belongs in the other discipline, which effectively leaves only the more generic (and usually less prestigious) "any inter-disciplinary research" journals.
That becomes a problem if institutions are focused on publication metrics and venues as a priority, as it means the institutional incentives drive people towards or away from these kinds of work. Ideally we need negative results published in the same venues as positive results (in my view), so those doing this valuable work don't get "relegated" to the negative results journals, which will no doubt get less attention at tenure review time, or from grant funders looking at publications etc.
One important task of the advisor is to filter bad projects. Does the paper pass the smell test? Are the authors reliable or they had published "dubious" results before?
[I was recently discussing a paper in a virtual meeting. My conclusion is that Zoom does not have enough negative reactions.]
Also, before assigning an experiment to a graduate student, try to have something like a plan:
1) This will surely work and is interesting enough for the dissertation and to be publish.
2) This part is dubious, because it's related to a too resent research paper or there may be some hidden technical problems that are difficult to see.
2') An underspecified alternative to 2, just in case.
3) Extra task for bonus points and perhaps a Nobel price.