News for hackers. Stuff that matters.
Re:Julia's Arraypocalypse Release by masklinn on Sunday September 27, 2015 @02:54PM (#10287487)
> Array slices will create views rather than copies. This is just a matter of performance

Except for breaking any existing code which mutates slices (unless slices are COW?), and implicit sharing can have disastrous effects on memory usage.

Re:Julia's Arraypocalypse Release by StefanKarpinski on Sunday September 27, 2015 @03:03PM (#10287521)
How can implicit sharing have "disasterous effects on memory usage"? It will certainly break some code, hence the large amount of warning and the "pocalypse" appellation. Note that this is how array slices work in NumPy and that ecosystem has not collapsed.
Re:Julia's Arraypocalypse Release by masklinn on Sunday September 27, 2015 @03:12PM (#10287553)
> How can implicit sharing have "disasterous effects on memory usage"?

A small long-lived slices keeping a original arrays alive implicitly. Oracle recently undid the same optimisation in Java's strings because of that exact issue (which obviously broke code the other way around, code relying on cheap substrings would now take O(n) instead of O(1)). Erlang developers are also warned early and often about that behaviour in binaries.

Re:Julia's Arraypocalypse Release by btown on Sunday September 27, 2015 @03:53PM (#10287717)
Very true. Less of a problem in scientific computing than web services, I think, since it's not common to keep slices around: you generally process a "frame" or an iteration of a simulation (Monte Carlo, etc.) and perhaps compare it to a previous frame, but the only history you keep beyond that are aggregates of the data, rather than individual slices that might go (say) in a global dictionary and stay around for a while. I don't think I've ever seen NumPy code where array slices were used as values in a long-living in-memory key-value store or cache... whereas you see that all the time with substrings in code for webservices. Thus it hasn't been as big a problem as in the Java community.

Still something that should be in bold text in Julia 0.5's readmes, though; slicing is cheap now, but it comes with this very important caveat!

Re:Julia's Arraypocalypse Release by johnmyleswhite on Sunday September 27, 2015 @04:16PM (#10287796)
The primary use case for Julia's coming array slices is to provide views into an array that you always intended to keep alive anyway.

Iterating over the columns of an in-memory matrix, for example, is the use case that this change is meant to speed up.

Re:Julia's Arraypocalypse Release by masklinn on Monday September 28, 2015 @04:04AM (#10289348)
And so it was with Java's substring and is for Erlang's binaries sharing. I will acknowledge that array slices are less likely to cause problems than string slices, but it is still a possible issue for the unwary. Which was my point.
Re:Julia's Arraypocalypse Release by StefanKarpinski on Sunday September 27, 2015 @04:34PM (#10287870)
Totally different typical usage patterns. Taking substrings and throwing away the original string is the norm for text. Taking a column of a matrix and throwing away the original matrix is not. Different domains call for different design tradeoffs – this change stems from extensive practical experience with writing numerical algorithms.
Re:Julia's Arraypocalypse Release by masklinn on Monday September 28, 2015 @04:06AM (#10289357)
Note that I am not saying it's a bad default (it still is the default of Erlang binaries).

I am saying this change is not quite "just a matter of performance", that the documentation and changelogs should be very very clear about the behavioural changes and the possible issues with the new behaviour, and that there should be an easy way to "unshare" slices and force a copy.

Re:Julia's Arraypocalypse Release by lqdc13 on Sunday September 27, 2015 @05:27PM (#10287991)
would there still be an easy way to make copies?

Like advanced indexing (http://docs.scipy.org/doc/numpy/reference/arrays.indexing.ht...) in the Python world?

Re:Julia's Arraypocalypse Release by elcritch on Monday September 28, 2015 @12:35AM (#10288930)
You could just use a list comprehension, e.g. `[ x for x in arrayveiw1 ]` or perhaps a copy function will be added to Base.
Re:Julia's Arraypocalypse Release by StefanKarpinski on Monday September 28, 2015 @10:43AM (#10290695)
There's already a copy function that does this.
Re:Julia's Arraypocalypse Release by elcritch on Tuesday September 29, 2015 @04:47PM (#10299515)
Great, Julia seems to have a good assortment of those "I need XYZ" (for scientific programming) and it's already in Base. I should check the apropos more.
Re:Julia's Arraypocalypse Release by sdevlin on Sunday September 27, 2015 @03:18PM (#10287582)
The post you're responding to says explicitly in the first sentence that these are breaking changes.
Re:Julia's Arraypocalypse Release by masklinn on Monday September 28, 2015 @04:01AM (#10289343)
1. the comment I replied to stated — and this is the part I quoted and to which I was responding — that slicing creating views is just a matter of performance.

2. there are gradations in breaking changes, the breaking changes I outline are insidious because they are silent, they won't cause the program to fail loudly, but they will cause it to behave very differently than it used to and possibly to corrupt data.