News for hackers. Stuff that matters.
Re:Plan to replicate 50 high-impact cancer papers shrinks to... by roel_v on Tuesday August 07, 2018 @02:04AM (#17704179)
It would require either all researchers to be 25% software engineers, or to add 30-50% headcount by adding SE's. It would be a radical change in methods and funding. There is massive institutional pushback to moving in this direction, as people already feel too much is done in/by computers. To make any research to be publishable depend on software engineering technicalities...

OTOH, it would be awesome for me, as I've been doing consulting on exactly this for 15+ years :)

Re:Plan to replicate 50 high-impact cancer papers shrinks to... by ajuc on Tuesday August 07, 2018 @02:28AM (#17704252)
Is it better to get the correct answers 25% slower, or incorrect answers at 100% speed?

How do you do science if you can't repeat experiments?

Re:Plan to replicate 50 high-impact cancer papers shrinks to... by roel_v on Tuesday August 07, 2018 @08:12AM (#17705741)
shrug What does 'the correct answer' even mean for any real world problem? There is so much we don't know. I'm not saying we should just make up papers and call it good, but there is a gradient between 'making stuff up' and 'doing everything according to a hypothetical ideal methodology all the time'. It's a matter of opinion how much not having a 100% repeatable experiment for each paper moves us towards the left of that gradient. It would be an improvement, yes, but I would put it more in the 'moves us from 75 to 80' ballpark than in the 'moves us from 20 to 100' that most people in this thread seem to believe.

"How do you do science if you can't repeat experiments?"

Well, the way we've been doing it for 300 years? It's not like the flawed processes of the past didn't yield anything. To put it in operations research terms: science doesn't work like hill climbing, it's more like a stochastic optimization process without a (well defined) halting condition. Again, I'm not saying we can't and shouldn't strive to improve, but the first year grad student level huff puffery in this thread is just completely detached from reality. 'Science' as you were explained the concept in high school is only a high-level abstraction of the concept, it's not how actual things are or get done in the real world.

Re:Plan to replicate 50 high-impact cancer papers shrinks to... by your-nanny on Tuesday August 07, 2018 @09:12AM (#17706215)
"first year grad student level huff puffery"

personally I find the worriers overstating the problem of reproducibility. lots of results are reproduced all the time, but in the course of testing other hypotheses.

if a result depends on exact replication of methods it isn't that robust an effect, and just might be a trivial one.

Re:Plan to replicate 50 high-impact cancer papers shrinks to... by ahartmetz on Tuesday August 07, 2018 @03:42AM (#17704542)
I have worked on experimental high-energy physics with computers and had the same idea - physicists should just hire software engineers. Large amounts of physicist time are spent cobbling together, and suffering from, shoddy software. At least we had public repositories with version numbers, and datasets were shared and uniquely named.
Re:Plan to replicate 50 high-impact cancer papers shrinks to... by your-nanny on Tuesday August 07, 2018 @09:12AM (#17706220)
that's my job!