News for hackers. Stuff that matters.
Re:Leela Zero – Go engine with no human-provided knowledge by dkersten on Thursday November 16, 2017 @05:45AM (#15711644)
> so the code is almost never published

Even worse is that a friend of mine who is a researcher in neuroscience/BCI has found that many of the papers couldn't be reproduced at all with flawed or incorrect conclusions. Since they typically don't publish their code, its very difficult for others to check the correctness.

Re:Leela Zero – Go engine with no human-provided knowledge by vog on Thursday November 16, 2017 @05:48AM (#15711664)
Moreover, in this context "code" is not just Python/C++/Java, but also spreadsheets!

Publishing spreadsheets is perhaps an even more urgent issue than publishing other types of code.

And of course the underlying data (anonymized) in case of statistical elicitation, but that's another topic. (Open Access, Open Data and Free Software are very closely related, though).

Re:Leela Zero – Go engine with no human-provided knowledge by fjfaase on Thursday November 16, 2017 @07:25AM (#15712121)
And even when it is made available, it is of very poor quality. I heard about someome who studied the programs used for analyzing DNA sequences. While tinkering with it, he found a bug, fixed it and submitted a pull request. Two years later, the pull request was still not merged by the maintainer of this 'standard' package. He also told that the software was of a poor quality. For example, in theory, four base pairs could be stored in one byte, instead they are simply storing the letters A,C,T, or G in a single byte for each pair.
Re:Leela Zero – Go engine with no human-provided knowledge by dkersten on Thursday November 16, 2017 @08:37AM (#15712492)
Yeah, that’s what my friend was saying too. He mentioned one often-cited paper which, when he fixed a bug he found in the code, no longer supported the papers conclusion at all. He also complained about the poor quality in a lot of the code. :-(
Re:Leela Zero – Go engine with no human-provided knowledge by znfii on Thursday November 16, 2017 @12:47PM (#15714746)
I haven't tested this myself, but from what I have heard it is not necessarily a good idea to store several different things in a single byte like this. While it may consume less memory there is a performance cost for each access since you may need to do masks and shifts.

Of course, there is a non-trivial performance trade-off with more cycles but more objects fits in cache, etc.

Some quick googling gave for example this source which seems to hint that under some situations it is not worth it at least https://stackoverflow.com/questions/4240974/when-is-it-worth...

Lastly, it seems like being quite harsh to call the code being of poor quality because of something like that. Unless it really is very important to get the extra memory, just the maintenance cost of packing things efficiently might make it not worth it. Even production code bases such as the JVM will happily use a single byte (or even more?) to store a boolean, and such things probably occur all over the place.

Re:Leela Zero – Go engine with no human-provided knowledge by kirillseva on Thursday November 16, 2017 @04:26PM (#15716779)
unless the author uses memory in the most efficient way possible (preferably writing LLVM-IR directly) it will be hard for reviewers to run DNA analysis algorithms on their smart fridges /s
Re:Leela Zero – Go engine with no human-provided knowledge by raverbashing on Thursday November 16, 2017 @06:24AM (#15711840)
Really, the academic world is sometimes obsessed with pseudocode.

It might make some didactic sense, but it makes fewer and fewer sense as times go by when you can ship the actual code and shows actual results

Re:Leela Zero – Go engine with no human-provided knowledge by deepnotderp on Thursday November 16, 2017 @01:38PM (#15715256)
Even worse is when you realize that's a growing concern in deep learning.