News for hackers. Stuff that matters.
Re:It’s OK for your open source library to be a bit shitty (... by ljm on Sunday May 17, 2020 @06:49AM (#23211096)
If anyone is in any doubt about this, I would like to point them towards this github issue: https://github.com/mrc-ide/covid-sim/issues/165
Re:It’s OK for your open source library to be a bit shitty (... by maest on Sunday May 17, 2020 @07:13AM (#23211189)
Some of the commenters in that issue have a very regressive and myopic stance. Raising such a public ruckus will heavily disincentivise scientists from publishing their code in the future. Decisions will still be made, but in the dark. There's a certain amount of self-stroking going on there with people feeling better by putting down code produced by non-professional programmers.

A much better response would be:

1. Scrutinise end users of the code: demand that governments only use this data if proper due dil has been carried out.

2. Submit code improvements.

3. (maybe) Demand that peer reviewers are more rigorous when checking the way results are generated. The issue with this is that this would also be a strong disincentive for scientist to publish their work.

Re:It’s OK for your open source library to be a bit shitty (... by dirtydroog on Sunday May 17, 2020 @10:09AM (#23212104)
Well, there was this situation a few years ago https://www.theverge.com/2013/4/17/4234136/excel-calculation...

While the tone of the covid model scrutiny leaves a lot to be desired it is understandable seeing as how the results are affecting the entire world. The situation is unprecedented.

Re:It’s OK for your open source library to be a bit shitty (... by maest on Sunday May 17, 2020 @11:06AM (#23212502)
I don't think anyone is saying the code can't be better or even that the results might be wrong (I don't know as I haven't spent much time going through the codebase).

My meta point is that people are:

1. pressuring and blaming the wrong party. 2. doings something which will have an unintended strongly negative consequence (make scientists averse to publishing their code).

Re:It’s OK for your open source library to be a bit shitty (... by guygurari on Sunday May 17, 2020 @07:08AM (#23211167)
I don’t doubt it, but covid-sim is not the best example of this effect. In this case, people are worried that poorly written simulation code is informing policy decisions that affect billions of people. They are not demanding that it should be fixed, they are demanding that it should not be relied on in its current state.
Re:It’s OK for your open source library to be a bit shitty (... by wizzwizz4 on Sunday May 17, 2020 @10:42AM (#23212343)
They're worried that there isn't testing code to prove that it's correct. If it's proven correct in other ways, it doesn't need unit tests.

Yeah, the code's bad – so what? It wasn't written by programmers. Most simulation code is bad, but if it's been proven correct it doesn't need to be good.

From what I can tell, this is a matter of conflicting conventions in different fields meeting head-on.

Re:It’s OK for your open source library to be a bit shitty (... by guygurari on Sunday May 17, 2020 @04:37PM (#23215235)
To be clear, I am not arguing for or against the validity of the claims regarding covid-sim. I am only saying that this is not a good example of the effect where people feel entitled to get open source code that fits their needs.
Re:It’s OK for your open source library to be a bit shitty (... by ohgodplsno on Sunday May 17, 2020 @08:54AM (#23211640)
Dunning-Kruger powered software developers with beliefs that they are in any way qualified to say that mathematical simulations checked by dozens of actual scientists with PhDs are not valid ask that results are ignored because there's no unit tests.
Re:It’s OK for your open source library to be a bit shitty (... by noelsusman on Sunday May 17, 2020 @08:46AM (#23211592)
I don't even know where to begin with this, it's so insane. It's extra funny because John Carmack was the one who did a lot of the refactoring of this code. It's also not remotely true that billions of lives have been disrupted on the basis of this code being correct.

This is exactly why scientists don't release their code.

Re:It’s OK for your open source library to be a bit shitty (... by tsbertalan on Sunday May 17, 2020 @03:45PM (#23214752)
Another reason scientists often don't release code is that code is not considered the valuable artifact produced by research--in more methods-focused fields in particular, it's instead the mathematical model, as embodied in some latex in the published paper. "Reproducing" the results then doesn't consist of trivially rerunning the same code, but in reimplementing it, possibly in a different language. This could maybe be seen as valuable in that the reproduction will surface both implementation errors and logic/modeling errors (though, really, having the source facilitates both).

In fact, more applications-focused researchers (those who combine real data with established models, for instance) tend to write higher-level scripts stringing more-engineered tools. In this case, open sourcing the scripts would be both easier and more pointless, since they will be almost the same as what is stated in English, tables, and plots in the paper. Epidemiology is usually this way, in my experience, though the linked repository seems to have some of both flavors.

The underlying misunderstand in that issue thread seems to me to be a disagreement on what the main valuable byproducts of scientific programming are. Professionally programmers will naturally think it's the code, but, traditionally, the code has been seen in academia as only a temporary vehicle for the model and it's parameters, which is the real product. (Also, the "program" might really need to include the lab, its papers, and its institutional knowledge, which is harder to package and open-source.)

Right or wrong, the assumption is that any competent grad student could reproduce the result in a week or so of (admittedly unnecessary) greenfield coding. But this is clearly not ideal, and newer work does strive for more professionalism in coding, open-source-by-default, and therefore faster replication. The project in question clearly predates this trend.

(Of course, a third reason academics don't open source is that some secrecy is required before publication in competitive fields. On a months-long project, you might not want to be committing publicly before publication. But this isn't much of an excuse.)

Re:It’s OK for your open source library to be a bit shitty (... by UncleMeat on Sunday May 17, 2020 @12:26PM (#23213027)
Yup. This whole experience was immensely frustrating. First people complain that the code isn't open source and as soon as it is available suddenly there are piles of people swooping in to shit on everything. That's not going to make scientists write better code. It is going to make scientists refuse to open their code.

The only thing more frustrating is when I see people swoop into open source code with "security vulns" that are based on nonsense threat models.

Re:It’s OK for your open source library to be a bit shitty (... by umvi on Sunday May 17, 2020 @11:13AM (#23212542)
The scientism is very strong in that thread.

Isn't "faith in science" an oxymoron? If anything, people should default to skepticism for hastily thrown together scientific models.

Re:It’s OK for your open source library to be a bit shitty (... by ww520 on Sunday May 17, 2020 @10:06AM (#23212074)
WOW. This is over the top toxic.
Re:It’s OK for your open source library to be a bit shitty (... by kirill5pol on Sunday May 17, 2020 @10:30AM (#23212265)
That was painful to read. :(