News for hackers. Stuff that matters.
Re:Correctness and composability bugs in the Julia ecosystem by wodenokoto on Monday May 16, 2022 @09:57AM (#31397272)
According to the article the problem is in the ecosystem, and partly the standard lib.

Basically it doesn't matter if Julia the language is fine, if all the stats packages make wrong calculations. Then what is the point of Julia, if you have to rewrite all things? might as well use another language where you trust the result of the ecosystem, since it is the ecosystem you need in order to produce results.

Re:Correctness and composability bugs in the Julia ecosystem by trenchgun on Monday May 16, 2022 @10:37AM (#31398006)
All bugs mentioned had been quickly fixed: https://news.ycombinator.com/item?id=31397425
Re:Correctness and composability bugs in the Julia ecosystem by wnoise on Monday May 16, 2022 @10:54AM (#31398296)
That comment doesn't say all bugs have been fixed, or even quickly fixed. When I check on the posted links, many are in fact still open, e.g.

https://github.com/JuliaStats/Distributions.jl/issues/1253

https://github.com/JuliaStats/StatsBase.jl/issues/642

https://github.com/JuliaStats/StatsBase.jl/issues/616

https://github.com/JuliaLang/julia/issues/39385

Re:Correctness and composability bugs in the Julia ecosystem by pdeffebach on Monday May 16, 2022 @11:25AM (#31398838)
I don't get these complaints about `sum!(a, a)`. Sure it's a bit of a footgun that you can overwrite the array you are working with. This doesn't rise to a "major problem" of composability.

The histogram errors seem annoying though. Hopefully they can get fixed.

Re:Correctness and composability bugs in the Julia ecosystem by wnoise on Monday May 16, 2022 @12:37PM (#31399936)
Sure, it's unsurprising that it produces unexpected results, but there are actually semantics that should be expected. The problem is that implementing those semantics correctly for all cases is hard, because aliasing. Same issue that e.g. memcpy() vs memmove() have.
Re:Correctness and composability bugs in the Julia ecosystem by pdeffebach on Monday May 16, 2022 @12:38PM (#31399964)
What semantics are expected in other languages? This seems solidly in the realm of undefined behavior as far as I can tell.
Re:Correctness and composability bugs in the Julia ecosystem by wnoise on Monday May 16, 2022 @01:27PM (#31400657)
The obvious semantics for these functions is that f!(a, args...) should do the same thing as a .= f(args...).

It's only undefined behavior because the simple implementations don't do that in the presence of aliasing.

I brought up memcpy() and memmove() (which in C are copying identity functions on bytes) exactly for this point. memcpy() has undefined behavior when the source and destination ranges overlap (implementable as a simple loop), while memmove() does the right thing if they do overlap, at the cost of having to check what direction they overlap when they do. And in C you can actually easily check if they overlap and in what direction, because the only interface there is the pointer. Aliasing with objects with internal details that are more complicated than that to check is difficult, perhaps too difficult to expect. But it is possible if your only handling your own objects: witness analogous behavior getting specified in numpy: https://docs.scipy.org/doc/numpy-1.13.0/release.html#ufunc-b... . They do note that this can require allocation, even in some cases where it shouldn't. But not allocating is of course most of the point of the in-place versions.

Re:Correctness and composability bugs in the Julia ecosystem by pdeffebach on Tuesday May 17, 2022 @09:56AM (#31410709)
Thanks for the detailed response.

Yeah allocation seems like the biggest hangup here. I would rather have a function stick to a "no allocating" contract and allow for some undefined behavior than have a function unexpectedly allocate to preserve safety.