News for hackers. Stuff that matters.
Re:Differences between the word2vec paper and its implementa... by p1esk on Monday June 03, 2019 @09:23PM (#20090857)
Pardon me, maybe I just misunderstood the whole idea of "reproducible" but it is only not reproducible if you tried to implement the ideas described in the paper yourself, and I mean really tried, and contacted the authors for help, and still failed to achieve the claimed results.
Re:Differences between the word2vec paper and its implementa... by throwawayjava on Monday June 03, 2019 @09:47PM (#20090995)
> if you tried to implement the ideas described in the paper yourself, and I mean really tried, and contacted the authors for help, and still failed to achieve the claimed results.

This is the actually important definition of "reproducible".

"has an install.sh" is really nice, but it's far more important that an informed reader can recreate the artifact for themselves from the written description.

The "must be able to apt install or it's not real science" is a particularly dangerous path to go down. It's how you end up with e.g. shit loads of well-engineered LISP or FORTRAN code with no actual scientific insights or knowledge transfer. Which has actually happened in the past. A lot.

Re:Differences between the word2vec paper and its implementa... by telchar on Tuesday June 04, 2019 @11:06PM (#20101153)
I don't think people in this thread are insisting on a downloadable package. Just that the algorithm published in the write-up accurately describe the work that was actually done to achieve the stated results. If even that threshold can't be met, IMO the authors are polluting the literature with bad work.
Re:Differences between the word2vec paper and its implementa... by derefr on Monday June 03, 2019 @11:41PM (#20091648)
> and contacted the authors for help

Isn’t the whole point of a paper, to obviate needing to talk to the authors? Like, so that science is a ratchet that doesn’t slip backward the moment the authors die?