News for hackers. Stuff that matters.
Re:How Should We Critique Research? by jasonhong on Friday April 16, 2021 @01:49PM (#26836998)
> If a study doesn’t replicate then it’s almost certainly nonsense.

For some fields, especially natural sciences, this statement makes sense. However, for sciences of the artificial (to use Herb Simon's term) I would respectfully disagree.

For example, sometimes the underlying context has changed. I remember a luminary in my field (HCI) once argued that we should consider re-doing many studies every decade, because each cohort uses a different set of tools and has a different set of experiences, and because the underlying demographics of those cohorts change.

Re:How Should We Critique Research? by infogulch on Saturday April 17, 2021 @12:07AM (#26841501)
This is what we should be paying people like rscho upthread to do. Graduate students should cut their teeth on half a dozen replication studies. Stop chasing novelty at all costs. Instead improve process, methodologies, statistical analysis, etc etc. Our system doesn't pay for that though, so instead we get to live in the world where researchers actively and knowingly publish junk so they can feed their family.