News for hackers. Stuff that matters.
Re:Most scientists 'can't replicate studies by their peers' by binaryzeitgeist on Thursday February 23, 2017 @11:51AM (#13715350)
Specifically to the CS context, I think some version of double blinded peer code-review should be made mandatory for a publication.

I've seen authors skip quite a many details that are quintessential to the replication process.

In short, if research is not replicable by the peer community it just useless, that's what it is.

Re:Most scientists 'can't replicate studies by their peers' by jedberg on Thursday February 23, 2017 @01:27PM (#13716341)
CS is the craziest of them all. Those should be the easiest to replicate. "Here is the code, here is a manifest of the environment/container/disk image/etc." You should be able to take that and run it and get the same results.

Or are you saying that the code itself is the problem and that they've done the equivalent of "return True" to get the result they want?

Re:Most scientists 'can't replicate studies by their peers' by tnecniv on Thursday February 23, 2017 @03:25PM (#13717418)
In my other comment I mentioned the CS results I've largely struggled to reproduce is because they include enough detail for you to get the gist of how it works, but not enough to avoid going down some rabbit holes. Also, not all publications include code. Many venues don't require it.