News for hackers. Stuff that matters.
Re:Please Commit More Blatant Academic Fraud by michaelt on Friday February 21, 2025 @06:39AM (#43126410)
> And we must ensure that explicit suggestions to modify one’s science in the service of one’s career – “you need to do X to be published”, “you need to publish Y to graduate”, “you need to avoid criticizing Z to get hired” – carry social penalties as severe as a suggestion of plagiarism or fraud.

One of the pernicious things in this area is that, even as we teach young researchers how to avoid making mistakes and engage sceptically with the work of others and that scientific fraud is a nontrivial issue, we also tell them how to commit fraud themselves and that their competition is doing it.

"Watch out for P-hacking, that's where the researcher uses a form of analysis that has a small chance of a false positive, and analyses loads of subsets of your dataset until a false positive arises and just publishes that one"

"Watch out for over-fitting to benchmarks, like a car taking the speed crown by sacrificing the ability to corner"

"Watch out for incomplete descriptions of test setups, like testing on a 'continent-scale map' but not mentioning how detailed a map it was"

"Watch out for citations where the cited paper doesn't say what is claimed, some people will copy-and-paste citations without reading the source paper"

"Watch out for papers using complicated notation, fancy equations and jargon to make you feel this looks like a 'proper' paper"

"Watch out for deceptive choice of accurate numbers, like a study with a 25% completion rate including the drop-outs in the number of participants"

"Watch out for simulations with inaccurate noise models, if the noise is gaussian in the simulation but a random walk in reality, great simulated results won't transfer to reality"

I've made no suggestion at all that you should modify your science or commit fraud - but I've also just trained you in how to do it.

Re:Please Commit More Blatant Academic Fraud by proto-n on Friday February 21, 2025 @06:53AM (#43126496)
It's really not that hard to come up with ways to commit fraud if you want to. On the other hand, it's very easy to make such mistakes if you don't know to avoid them. This characterization is very misguided IMO.
Re:Please Commit More Blatant Academic Fraud by michaelt on Friday February 21, 2025 @08:15AM (#43127048)
Ah, perhaps I wasn't clear about what I'm trying to say. I don't think we should stop training researchers in common mistakes and fraudulent methods to watch out for.

I'm just saying: I don't believe anyone actually tells budding researchers that they should commit fraud. Instead I think the process is probably more like this:

Year 1: Statistics/research training. Here are a load of subtle mistakes to watch out for and avoid. Scientific fraud happens sometimes. Don't do it, it's very dishonest.

Year 2: Starting research. Gee a lot of these papers I'm reading are hard to reproduce, or unclear. Maybe fraud is widespread - or maybe they're just smarter or better equipped than me.

Year 3: "You really ought to have published some papers by now, the average student in your position has 3 papers. If you don't want to flunk out you really need to start showing some progress"

Re:Please Commit More Blatant Academic Fraud by proto-n on Friday February 21, 2025 @12:37PM (#43130428)
I still disagree. It's more like "omg I should have published at least a few papers by now, what am I doing" and then you start frantically looking for provable things in the dataset. You find one that you can also support with a nice story. Now either a) You were not tought about how or why this is wrong and you publish the paper b) You were, and know that you should collect a separate dataset to test the hypothesis. But also, there is a huge existential pressure to just close your eyes and roll with it.

It's not that you need to be tought how to cheat, it's that you need to be tought how to avoid unintentionally cheating.

Re:Please Commit More Blatant Academic Fraud by genewitch on Friday February 21, 2025 @05:19PM (#43133743)
Maybe I'm misreading but you both seem to say the same thing.

New student is shown how to read a paper, how to spot egregious errors and all the things listed above.

Student, i guess feels forced to publish. And maybe uses murkier tactics to get the paper published.

As Dr Frank Etscorn said, "I can show anything correlates to anything else." We were discussing vitamin D papers anf I was testing that paper funding AI mentioned on HN a few times last year.

Re:Please Commit More Blatant Academic Fraud by genewitch on Friday February 21, 2025 @10:06PM (#43135698)
Jeez, i apologize. "testing that 'paper-finding' AI mentioned"
Re:Please Commit More Blatant Academic Fraud by genewitch on Friday February 21, 2025 @05:20PM (#43133749)
I can't edit from my phone, maybe the nuance is in your final sentence?