dfeliej

In my experience, there's a disproportionate reward associated with fashions in science. That is, the people at the forefront of a trend get rewarded professionally (tenure, positions, promotion), even when behaving in a way that is irresponsible theoretically or empirically. The skeptics, on the other hand, who carefully lay out the problems with the new research, and eventually point out the obvious flaws, get treated as sticks-in-the-mud, etc., even though they are eventually proven correct.

There are gains to be made through innovation, and I'm not arguing against that. Sometimes too, details are just minor details to work out. But sometimes the details matter and point to bigger problems.

The problems with fads in science are that the focus in academics seems sometimes to be on popularity per se rather than verdicality, and denialism about scientific fads as a phenomenon. This is pointed out in the article, and is one of its better points.

Someone else criticized the article for not presenting solutions, which is fair, but I think that's in part because effective solutions would probably require a huge shift in culture in academics. We'd need to return to more stable professional positions rather than the cannabilistic postdoc system in place now, with declining tenure; we'd need to focus more on groups of researchers and findings rather than celebrity academics ala TED talks; we'd need to put much more emphasis on a pattern of replications rather than single studies; etc. etc. Even then, I don't think the problem would entirely go away.

Some of the research in this area is misleading, I'd even maybe say poorly conducted, and I worry it will be taken too much at face value by policy-makers and administrators.

The conclusion that institutition doesn't matter, for example, seems ridiculous and flies in the face of a lot of IO research. People move for reasons, for example; the question isn't whether productivity changes after a move on average, it's what would have happened to an individual researcher if they had stayed in a given situation relative to what would have happened after a move to another situation. There's a lot of cliques, the moves aren't random, etc.

It's a bit like concluding that you can determine the importance of a paper solely by looking at the impact factor of a journal.

I read the J Clin Epi paper being discussed when it first came out and it expressed my feelings and thoughts very well, as a professor.

The crux of the problem is this: we as a society value science, and the epitome of this in healthcare has been the goal of evidence based medicine (EBM) or evidence based practice (EBP). The idea seems simple enough, that healthcare decisions and procedures should be based on scientific evidence.

This clashes, however, with the reality that much of research in the biomedical sciences is unreplicable or completely invalid for other reasons. The reasons for this are complex but you have things like p-hacking, selective reporting of results, confounds, and so forth and so on. Think "how to lie with science" instead of "how to lie with statistics."

As a result, EBM gets hijacked by those who are trying to manipulate; appeals to science becomes a form of fallacious argument, sort of a version of appeal to authority error, where one appeals to a scientific finding because it's peer reviewed etc. without invoking the underlying integrity of the research per se. Then you have the unscrupulous making money off of bad science, and authority figures (in medical care and administration) cudgeling people for not using EBM/EBP when what is really happening is people are recognizing it as bad science. Sometimes no science is better than bad science--or sometimes, science means rejecting peer-reviewed research, even with all the right surface features.

The paper is part of a slightly personal exchange between Ioannidis and someone he looks up to but is also close to, so it has a kind of poignant conversational style. It's not so much a scientific paper as it is a personal discussion.

I came to suggest something similar.

I often wonder how much this applies today. Academic science is currently heavily driven by funding of popular, low-risk research.

Also, without meaning any disrespect to Feynman or the advice, it's easy enough to say something like "disregard what others are doing" when you've won a Nobel. It's another thing to say this to people in just about any other inequivalent scenario, which is nearly everyone.