News for hackers. Stuff that matters.
Re:'Serious errors' in research linking deaths to red meat by pvaldes on Monday September 05, 2022 @10:44AM (#32724869)
> what needs to change in the education process so studies are better designed

Remove politics, free kidnapped and gagged science.

Re:'Serious errors' in research linking deaths to red meat by mumblemumble on Monday September 05, 2022 @11:29AM (#32725377)
Politics and education are certainly factors, but I would argue that they are minor. The bigger issue is the natural incentive structures that academia has created. If you want to have a career, you get both attention and funding in return for headline-grabbing results. This creates two huge second-order effects: first, everyone's incentivized to look for big impressive new effects. Second, focusing on confirmatory work is a great way to tank your career. And those create a third order effect: people who cut corners thrive, and there's not much to keep that in check because people who cautiously vet their work and others' get stuck in adjunct professor hell until they finally give up and exit the field.

Politics just adds some spin onto that. Education, I just don't think that's even a possible solution. The math involved in some of these fancier statistical techniques can be devilishly hard to get right, and it's often simply asking too much to expect someone to be an expert at both that and molecular biology. You could argue that that means people should look for competent co-authors to cover that side of things. But, see above - current incentive structures don't really favor the researcher who says, "I know! Let's bring on some nerd whose only job is to find reasons why we shouldn't publish!"

Re:'Serious errors' in research linking deaths to red meat by pvaldes on Monday September 05, 2022 @12:00PM (#32725796)
If we assign grant points to people that flood the precious journal space with fake articles claiming over-hyped promises, maybe we should give grant points also to the people able to spot the lies on those kind of articles.

After all they are serving to the advance of science also (and saving thousands of expensive hours of research to other). If you want to promote it, just create incentives.

Otherwise, If the cheater wins in a 95% of the times and the verifier does not have anything to gain (and will damage their own research by the time invested on this) after a while only cheaters remain [1]. At middle term, all research tend towards a sort of expensive shamanism in well illuminated white rooms.

[1] (And of course the non cheaters will be actively blocked from this point).

Re:'Serious errors' in research linking deaths to red meat by pvaldes on Monday September 05, 2022 @12:20PM (#32726043)
My proposal would be: If there was a clear, proven, intention of cheating and not just an error (we all commit stupid errors sometimes), the letter or communication of the people disclosing it will receive the same citation rank that the fake article had accumulated until that date. Good for the journals, good for the research.
Re:'Serious errors' in research linking deaths to red meat by MPSimmons on Monday September 05, 2022 @12:32PM (#32726192)
Like a bug bounty for academic publications?
Re:'Serious errors' in research linking deaths to red meat by pvaldes on Monday September 05, 2022 @12:44PM (#32726316)
Yes, something like that
Re:'Serious errors' in research linking deaths to red meat by WalterBright on Monday September 05, 2022 @02:04PM (#32727227)
> Politics and education are certainly factors, but I would argue that they are minor

Are they? If your proposed project does not align with current politics, no funding for you.

Re:'Serious errors' in research linking deaths to red meat by mike_hearn on Monday September 05, 2022 @02:22PM (#32727454)
From reading papers of various different types in the past few years it really depends a lot on the exact field in question.

Nutrition? Would be surprising if there's much political distortion there but there's lots of really bad methodologies, partly because it's hard to get good data (people are bad at reliably writing down what they eat).

Social bot/misinformation research? Basically just politics masquerading as science. Methodologies are invariably bogus and conclusions are clearly determined before any 'research' is done.

Epidemiology? A mix of bad incentives and ideological distortion. The focus on policy advice isn't really scientific, and their policies somehow always end up requiring massive state intervention in people's private lives. This isn't something that falls naturally out of the science (which is unreliable anyway) but more the fact that they don't even consider any other possibilities.

The above three are pretty bad fields. There are plenty of better ones. Computer science is mostly OK. AI and cryptography are slightly dubious. Stuff like databases or computer graphics are fine. So IMO it doesn't make sense to talk about research or academia in general, or it does but only when discussing the universal incentive problems. Some fields just seem naturally resistant to political game playing because they don't have much/any impact on social policy to begin with. Obviously I mean politics as in democratic politics here, not the more mundane everyday departmental politics.

Re:'Serious errors' in research linking deaths to red meat by cryptonector on Monday September 05, 2022 @04:50PM (#32729276)
Academia is pretty corrupt or corruptible (well, it's humans after all). If CS research isn't corrupted at the moment, it will be if it being corrupted becomes useful.
Re:'Serious errors' in research linking deaths to red meat by mumblemumble on Monday September 05, 2022 @09:41PM (#32731655)
I don't know about hard CS, and I don't know about that word "corrupt" because it implies malice and I think that Hanlon's Razor is probably a useful heuristic here. But, in machine learning, I've see plenty of the kind of thing I'm talking about. Papers often introduce new algorithms or techniques whose improvement over the competition can plausibly be attributed to excessively tuned hyperparameters that overfit the data on one or more benchmarks. It's really not all that far removed from p-hacking in my opinion.
Re:'Serious errors' in research linking deaths to red meat by wcarss on Monday September 05, 2022 @02:29PM (#32727544)
as has been the case in many, if not most, times, places, and contexts.
Re:'Serious errors' in research linking deaths to red meat by seventytwo on Monday September 05, 2022 @11:07AM (#32725150)
Politics is an emergent property of human interaction. It cannot be removed.
Re:'Serious errors' in research linking deaths to red meat by karaterobot on Monday September 05, 2022 @11:02AM (#32725087)
I think that's a separate issue that would not have affected this particular case.
Re:'Serious errors' in research linking deaths to red meat by xhkkffbf on Monday September 05, 2022 @11:15AM (#32725246)
That's just hard to do with a topic like this. There are committed people on both sides of the issue who know what they want to hear. That's not a good environment for science to find a neutral answer.
Re:'Serious errors' in research linking deaths to red meat by dapf on Monday September 05, 2022 @10:55AM (#32724995)
> Remove politics

That needs the remotion of taxation. I agree.