Credibility is one heuristic that let’s us figure out what might be worth investing time into, since your answer cannot be “everything”.
With respect to publishing, there is a significant volume of work, and we should be able to keep up with it long as people review as much as they publish.
That tweet was made before the report was released, but there were a few press stories about it. The second thread opens with a link to the report.
>Peer review is a BS credibility threshold that validates tons of garbage research in a way that financially benefits publications.
And this paper is of such low quality they couldn't even manage that. It has no merits, and even the paper admits the data is shit and the conclusion is unfounded.
Peer review is the process by which that judgment is made.
Criticizing peer review is a little bit like criticizing democracy. It's the worst possible way to go about doing things, except for all of the others we've tried.
Not if you are yourself a scientist.
Peer review is a social method of establishing the validity of research, but it's not the only method. Another would be collaboration between well-credentialed investigators. Yet another is having the backing of a large-scale professional firm or institution.
Peer review is lauded by people who either (A) don't know anything about research, or (B) do research without stellar credentials or the imprimatur of a large institution. Its chief proponents are the publications and conferences themselves, whose entire business model depends upon a perception of the elevated status of peer review.
Plenty of fictitious and fraudulent claims have been validated by peer review. https://en.wikipedia.org/wiki/List_of_scholarly_publishing_s...
If you want to review the research, read it yourself and evaluate the methodology.
What is a "well-credentialed" Investigator if it's not measured by peer review? It seems to me like an argument from authority, which is ten thousand times worse than "problematic peer review" you claim. Define well-credentialed without peer review without a cycle please.
What is "backing of a large-scale professional firm or institution"? They already exist, they are called journals, which perform, surprise, peer reviews. Other alternatives you mention are probably private organizations, you seriously believe it's less problematic than already less problematic journals? Much naivety in this argument.
Peer review has its problems, but as you've been told before, it's the best we have come up with. Your proposals are much worse than what we have.
If you are a "brand name" researcher (i.e. already well known and respected by your peers) peer review is even less meaningful as it's pretty much a rubber stamp.
It's not like many of these fields are just starting up.
https://fantasticanachronism.com/2020/09/11/whats-wrong-with...
Studies that don't replicate are cited at the same rate as those that do:
https://fantasticanachronism.com/2020/09/11/whats-wrong-with...
This year I've read a lot of epidemiology papers, and sometimes their peer reviews. There's something deeply wrong with peer reviewers in this field because they often write long, detailed reviews that completely ignore glaring problems in the papers, problems that jump out to 'lay' readers on the first glance through. My guess is that there are very complicated sets of unwritten rules about the sorts of problems that are and are not legitimate to criticise in peer reviews, and problems that are a little bit too fundamental, of the form "this entire paper should be rejected out of hand", don't get given when a paper has 25 authors at prestigious universities, even if the methodology or conclusions are absurd.
But in fields that the method is closer to science (e.g. physics, chemistry, neuroscience), I would expect that the overall field is converging to the truth and that senior authors will therefore be more tuned in to the best estimate of truth or how to get to it than junior authors.
Even in microbiology there are a huge number of papers that don't replicate.
To me it looks like the problems are general. They aren't restricted to a small set of social sciences.
While plenty of bad claims have been validate by peer review, it doesn't mean that peer review doesn't massively reduce the number of outrageously incorrect claims.
In any case, as it stands, the NAS article wouldn't pass peer review, because it makes borderline unphysical claims without providing enough evidence, even though doing so is quite easy. As for stellar credentials or the imprimatur of a large institution, one of the authors is fairly kooky and the institute itself has a massive conflict of interest.
There's a reason even the State Department is distancing itself from this thesis.
* Whatever skepticism you have toward NAS, I have a thousand times more skepticism toward a tweet thread from Sharon Weinberger.
* Peer review is a BS credibility threshold that validates tons of garbage research in a way that financially benefits publications. The peer review process is not itself a form of research. The paper should be judged on its own merits and the credibility of its authors.
* Contrary to what Weinberger said, a 64-page NAS document is available at https://www.nap.edu/read/25889/chapter/1