News for hackers. Stuff that matters.
Re:The Materials Project by infogulch on Monday July 04, 2022 @04:46PM (#31981587)
That Chemist tried to use a paper published in Nature to synthesize something for his lab, but he found a number of potential concerning details in the paper and was unable to reproduce it using the published method. The authors and publication ghosted him and the paper was still up after 2 months. https://www.youtube.com/watch?v=-WPBtFTLZnM

How do projects like this deal with papers published based on falsified data? Do they reproduce any of the source data themselves?

Re:The Materials Project by mkhorton on Monday July 04, 2022 @04:57PM (#31981688)
> How do projects like this deal with papers published based on falsified data? Do they reproduce any of the source data themselves?

I can't speak to this specific instance, but Materials Project does try to pay close attention to questions of reproducibility and provenance. Materials Project runs open-source repos[0] so that its methods can be verified, individual calculations are available via an API[1] and we also partner with NOMAD[2] to make larger files and calculation artifacts available for direct download. This is in addition to documenting methods via peer-reviewed papers, online docs, etc.

This is not to say that issues of reproducibility don't still exist, or that we ourselves couldn't be doing better. It's a big problem in the community.

[0] https://github.com/materialsproject [1] https://api.materialsproject.org/docs [2] https://www.nomad-coe.eu

Re:The Materials Project by dekhn on Monday July 04, 2022 @06:02PM (#31982197)
Retraction watch.

Btw many people (even trained) fail to reproduce legitimate work. It may not actually be fake or falsified, typically not being able to reproduce an experiment is insufficient evidence for retraction. I didn't watch the YouTube video.

Re:The Materials Project by nl on Tuesday July 05, 2022 @12:19AM (#31984271)
Can confirm.

People outside academia seem to believe that reproducibility is an explicit goal when publishing papers. Perhaps it should be, but generally speaking it isn't. Instead a paper is a record of the things the authors found interesting and novel when doing the work.

Re:The Materials Project by throwoutway on Tuesday July 05, 2022 @08:23AM (#31987353)
And if they’re objectively wrong, then what?
Re:The Materials Project by nl on Tuesday July 05, 2022 @10:05PM (#31995968)
"Wrong" is different to "fraudulent"

If they are wrong then you publish a paper with conflicting results. If there is fraud (which is different to "I can't replicate the results") then you deal with the publisher of the paper.

Re:The Materials Project by infogulch on Wednesday July 06, 2022 @02:50PM (#32005183)
Well there's two problems, first is journals refusing to publish anything that conflicts with previous publications (a despicable practice that goes against the most fundamental principles of science that include curiosity and the willingness to be wrong and change, see also [1]), second is a reasonable expectation to get a response from the original authors when asked about a failure to replicate (even in the mode of "you're stupid and did it wrong").

[1]: The importance of stupidity in scientific research | https://news.ycombinator.com/item?id=31977918