News for hackers. Stuff that matters.
Re:I've reproduced 130 research papers about “predicting the... by sgt101 on Saturday August 17, 2019 @03:13AM (#20722666)
I think you are a quick study; typically it takes me a week to figure out the detail of what's being done in a paper. Getting the data and testing would take longer. Charitably he/she has a framework with all the data required sitting ready to go and is just writing wrappers to the models. But even downloading frameworks from Github and getting them working takes a couple of days - for me. For example I've been playing with the Graph-network code from deepmind for a few weeks - I had to learn how the graphs were represented, how to build them and access them and how the models were made and put together. Just working that out was a solid three day job. Now I can build things and test out what's going on in the examples and get a feel for the framework, probably (if there was a problem) I would be in a reasonable position to say "this doesn't work like they think it does" (it does, but no surprise) but unless you've done that leg work I think you can't really. I think a proper replication effort is really 1 man month of expert time - or really you're just throwing stones.

There are no comments on this comment yet.