News for hackers. Stuff that matters.
Re:How Should We Critique Research? by exmadscientist on Friday April 16, 2021 @01:58PM (#26837114)
In practice, there's a massive bikeshedding problem in scientific review. It is very frustrating to see proposals or preprints criticized for missing a dot in paragraph 37 when the bigger problem is that the experiment is overall worthless because it can never change a decision.... This is bad enough when it's an honest mistake, but then there are all the dishonest or plainly incompetent people to worry about.

So, no, in fact I think we are suffering from an abundance of nitpickers. Science desperately needs more reviewers who can see the bigger pictures.

Re:How Should We Critique Research? by mike_hearn on Friday April 16, 2021 @02:04PM (#26837182)
That's fair enough. I was meaning nitpickers about logical fallacies specifically, not stuff like grammar or minor details of an experiment.

That said, is peer review really the place to dunk on a study because the whole goal is pointless? The work is done by that point, it's too late. It's the granting bodies that should be getting peer reviewed in that regard, but one of the root causes of the malaise in research is that the granting bodies appear to be entirely blind buyers. They care about dispersing money, not what they get out of the spending. If they didn't spend the money they'd be fired, so that's understandable. The core problem IMHO is the huge level of state funding of research. The buck stops at politicians but they are in no position to evaluate the quality of academic studies.

Re:How Should We Critique Research? by exmadscientist on Monday April 19, 2021 @11:35AM (#26863925)
I was actually thinking of grant applications when I wrote that.

And it's not even politicians, it's grant committees too. For the usual reasons, they'd rather fund study #13 from a researcher than anything that is the slightest bit uncomfortable to them.