News for hackers. Stuff that matters.
Re:A Code Glitch May Have Caused Errors in More Than a Hundr... by boulos on Saturday October 12, 2019 @02:27PM (#21235151)
It’s too bad that the code and correction seem to be locked down, but the linked Twitter discussion has an explanation [1]: the code expects Python’s glob to return a sorted list of files, but it doesn’t guarantee this (the main fix is to sort the result). I had thought reading the Vice article that this would be about case-sensitive vs insensitive filesystems.

[1] https://mobile.twitter.com/bmarwell/status/11818211438352015...

Re:A Code Glitch May Have Caused Errors in More Than a Hundr... by smnrchrds on Saturday October 12, 2019 @02:38PM (#21235228)
Are they locked down? The article links to the paper [0], which has two zip files to download, first of which is the code, including an explanation of the corrections made [1].

[0] https://pubs.acs.org/doi/10.1021/acs.orglett.9b03216

[1] https://pubs.acs.org/doi/suppl/10.1021/acs.orglett.9b03216/s...

Re:A Code Glitch May Have Caused Errors in More Than a Hundr... by boulos on Saturday October 12, 2019 @02:55PM (#21235365)
Ahh, I failed to click those. They seemed large enough to be “not the code, but the full dataset”. Thanks for the correction!
Re:A Code Glitch May Have Caused Errors in More Than a Hundr... by contravariant on Saturday October 12, 2019 @02:36PM (#21235215)
To me this just raises more questions like "Why does the order of the files even matter?", and "If it does why wouldn't you make sure it was sorted, or better yet sort it in some reliable fashion?".
Re:A Code Glitch May Have Caused Errors in More Than a Hundr... by lupire on Saturday October 12, 2019 @02:55PM (#21235363)
If you are writing a paper about the order in which a sequence of events happens, the order matters. It's the difference between cause and effect, the heart of science. Filename is often used as part of the data, as a timestamp.

Obviously reliable sorting is important, which is why people are upset that python's glob() turned out to be unreliable

Re:A Code Glitch May Have Caused Errors in More Than a Hundr... by magicbuzz on Saturday October 12, 2019 @03:32PM (#21235639)
IIRC glob.glob() is a wrapper around os.listdir(). I used to develop code on an OS X laptop then push to a Linux box for the actual computational result (big dataset). And discovered at one point that one OS returns a sorted result, the other does not. Sorting is important if you want track/analyse instrument data over the days in a year.
Re:A Code Glitch May Have Caused Errors in More Than a Hundr... by luckylion on Saturday October 12, 2019 @03:25PM (#21235586)
> Obviously reliable sorting is important, which is why people are upset that python's glob() turned out to be unreliable

A quick glance at the documentation didn't mention it, does glob even promise sorting the result? Relying on that to happen when it's not even promised seems negligent.

Re:A Code Glitch May Have Caused Errors in More Than a Hundr... by perl4ever on Saturday October 12, 2019 @03:39PM (#21235671)
I mean, I have learned that people simply won't accept that SQL doesn't guarantee the order of a result set if you don't use "order by". I tried to argue it and eventually gave up. It seems vastly easier to change a language than human nature. People are hardwired to ignore claims that something should not be relied on, because in normal human relations, that is a way of pushing away undue responsibility, not something to be taken literally.
Re:A Code Glitch May Have Caused Errors in More Than a Hundr... by bmc7505 on Saturday October 12, 2019 @05:09PM (#21236346)
> A quick glance at the documentation didn't mention it

What do you mean? It is the very first sentence of the documentation: "results are returned in arbitrary order"

https://docs.python.org/2/library/glob.html

Re:A Code Glitch May Have Caused Errors in More Than a Hundr... by luckylion on Saturday October 12, 2019 @05:34PM (#21236514)
Ah, see, I missed that, I had just skimmed, then grepped for "sort" and "order" (but mistyped that as oder).
Re:A Code Glitch May Have Caused Errors in More Than a Hundr... by magicbuzz on Saturday October 12, 2019 @03:35PM (#21235656)
Most developers look at the result from a function call rather than looking at the documentation.
Re:A Code Glitch May Have Caused Errors in More Than a Hundr... by emmelaich on Saturday October 12, 2019 @06:45PM (#21236887)
It doesn't sort. But people assume it does especially if the files are returned in the order that they were created, which typically will be sorted.

I think this is why it doesn't get noticed; the files are written say

   result-20190901-1303
   result-20190901-1305
   result-20190901-1310
   etc...