News for hackers. Stuff that matters.

A comment on Challenge to scientists: does your ten-year-old code still run?. Parent comment

Re:Challenge to scientists: does your ten-year-old code stil... by throwawaygh on Monday August 24, 2020 @12:36PM (#24262316)
> or in computer science / applied math/stats / etc., with different codebases, with different model variants, on different datasets) and the overall conclusions hold

A lot of open sourced CS research is not reproducible.

"the code still runs and gives the same output" is not the same as reproducibility.

[ Parent | on HN ]
Re:Challenge to scientists: does your ten-year-old code stil... by throwaway287391 on Monday August 24, 2020 @01:01PM (#24262664)
> A lot of open sourced CS research is not reproducible.

I'm not sure if this was meant to be a counter-argument to me, but I completely agree!

> "the code still runs and gives the same output" is not the same as reproducibility.

Yes, bit-for-bit identical results are neither necessary nor sufficient for reproducibility in the usual scientific sense.

[ Parent | on HN ]
Re:Challenge to scientists: does your ten-year-old code stil... by throwawaygh on Monday August 24, 2020 @02:25PM (#24263725)
> I'm not sure if this was meant to be a counter-argument to me

It wasn't :)

[ Parent | on HN ]