I wish there was more praise for negative results in publication, because whether confirmed or not, the knowledge has value.
It's hard to argue that I had justification to think that novel intervention X would have an effect. It turns out it doesn't. Science is often very specialized, there's little chance others would have the very same idea. If it works out, the argumentation would have to be reversed: my idea was very novel and non-obvious but as I show it actually works, which no one would have guessed.
The negative result story only works if the research community would have very strongly expected to see the effect, almost reversing the role of the null and the alternative.
Depends what you mean by "works". If you mean "is reasonably publishable in the current academic climate, then I agree. If you mean "has value", then I disagree.
In particular as that would mitigate problem #3 above ("3. The third, and hardest to solve, problem is not factoring in the whole population of experiments.")
I wrote an analysis of their data here: https://www.baybridgebio.com/blog/aducanumab-analysis.html
#1 and #2 are difficult to solve, given the near-exponential growth of funding for academic research.
https://www.researchgate.net/figure/Evolution-of-the-number-...
Could you show any data that supports the claim about nearly exponential growth of funding for academic research?
AFAIK the funding, especially considering growing number of people in the field, gets rather worse than better.
That seems like a really good idea, that would solve a lot of problems. Can we do that, please?
1. The first and most egregious type is outright fraud. This is when you intentionally manipulate or fake data. Everyone agrees this is bad, and honest actors are enough to prevent it. In some cases, other honest actors are sufficient to determine if the claims are fishy.
2. The second more subtle type is not paying attention to adaptivity. For example, maybe an investigator wants to look at the data before coming up with a hypothesis to test. This is dangerous because the investigator is already overfitting, so any p-values the investigator computes afterward do not mean what they're supposed to mean. This is less egregious because it's easy to do this just by not being careful or not knowing your statistics very well. A scientist can be honest, but imperfect, and do this. It's also not easy to sniff this out as a reviewer -- the scientist might just omit all the stuff that didn't work. But there appears to be growing awareness of this kind of problem.
3. The third, and hardest to solve, problem is not factoring in the whole population of experiments. This is where 100 labs independently try an idea and one of them gets a genuine (from their limited view) result with a genuine p-value. It's novel and that lab has (in the limited view) been careful about adaptivity and keeping their hypotheses carefully generated. Maybe they've even used carefully generated noise to ensure their conclusions generalize [1] (which would definitely cut down on this problem). So it's pretty much impossible for a reviewer to tell there's a problem, because they don't know about the 100 other people that tried this and failed because the randomness didn't go their way. Short of a public experiment registry, this one is hard to fix, especially because it may be that nobody's being malicious or ignorant.
[1] https://arxiv.org/abs/1411.2664