News for hackers. Stuff that matters.
Re:Correctness and composability bugs in the Julia ecosystem by ur-whale on Monday May 16, 2022 @09:53AM (#31397192)
The examples provided feel more like bugs in various libraries than an actual problem intrinsic to Julia the language.
Re:Correctness and composability bugs in the Julia ecosystem by wodenokoto on Monday May 16, 2022 @09:57AM (#31397272)
According to the article the problem is in the ecosystem, and partly the standard lib.

Basically it doesn't matter if Julia the language is fine, if all the stats packages make wrong calculations. Then what is the point of Julia, if you have to rewrite all things? might as well use another language where you trust the result of the ecosystem, since it is the ecosystem you need in order to produce results.

Re:Correctness and composability bugs in the Julia ecosystem by trenchgun on Monday May 16, 2022 @10:37AM (#31398006)
All bugs mentioned had been quickly fixed: https://news.ycombinator.com/item?id=31397425
Re:Correctness and composability bugs in the Julia ecosystem by wnoise on Monday May 16, 2022 @10:54AM (#31398296)
That comment doesn't say all bugs have been fixed, or even quickly fixed. When I check on the posted links, many are in fact still open, e.g.

https://github.com/JuliaStats/Distributions.jl/issues/1253

https://github.com/JuliaStats/StatsBase.jl/issues/642

https://github.com/JuliaStats/StatsBase.jl/issues/616

https://github.com/JuliaLang/julia/issues/39385

Re:Correctness and composability bugs in the Julia ecosystem by pdeffebach on Monday May 16, 2022 @11:25AM (#31398838)
I don't get these complaints about `sum!(a, a)`. Sure it's a bit of a footgun that you can overwrite the array you are working with. This doesn't rise to a "major problem" of composability.

The histogram errors seem annoying though. Hopefully they can get fixed.

Re:Correctness and composability bugs in the Julia ecosystem by wnoise on Monday May 16, 2022 @12:37PM (#31399936)
Sure, it's unsurprising that it produces unexpected results, but there are actually semantics that should be expected. The problem is that implementing those semantics correctly for all cases is hard, because aliasing. Same issue that e.g. memcpy() vs memmove() have.
Re:Correctness and composability bugs in the Julia ecosystem by pdeffebach on Monday May 16, 2022 @12:38PM (#31399964)
What semantics are expected in other languages? This seems solidly in the realm of undefined behavior as far as I can tell.
Re:Correctness and composability bugs in the Julia ecosystem by wnoise on Monday May 16, 2022 @01:27PM (#31400657)
The obvious semantics for these functions is that f!(a, args...) should do the same thing as a .= f(args...).

It's only undefined behavior because the simple implementations don't do that in the presence of aliasing.

I brought up memcpy() and memmove() (which in C are copying identity functions on bytes) exactly for this point. memcpy() has undefined behavior when the source and destination ranges overlap (implementable as a simple loop), while memmove() does the right thing if they do overlap, at the cost of having to check what direction they overlap when they do. And in C you can actually easily check if they overlap and in what direction, because the only interface there is the pointer. Aliasing with objects with internal details that are more complicated than that to check is difficult, perhaps too difficult to expect. But it is possible if your only handling your own objects: witness analogous behavior getting specified in numpy: https://docs.scipy.org/doc/numpy-1.13.0/release.html#ufunc-b... . They do note that this can require allocation, even in some cases where it shouldn't. But not allocating is of course most of the point of the in-place versions.

Re:Correctness and composability bugs in the Julia ecosystem by pdeffebach on Tuesday May 17, 2022 @09:56AM (#31410709)
Thanks for the detailed response.

Yeah allocation seems like the biggest hangup here. I would rather have a function stick to a "no allocating" contract and allow for some undefined behavior than have a function unexpectedly allocate to preserve safety.

Re:Correctness and composability bugs in the Julia ecosystem by exyi on Monday May 16, 2022 @09:57AM (#31397275)
Yea, all are just bugs, not some intrinsic flaws in the language.

Given Julia's goals (performance, abstractions, accessible to science people), it's understandable if they had slightly higher bug concentration than other (similarly sized) ecosystems.

Re:Correctness and composability bugs in the Julia ecosystem by kllrnohj on Monday May 16, 2022 @10:33AM (#31397927)
The author's argument is that the bugs all share a pattern, and thus there is an intrinsic flaw. That doesn't necessarily mean the community wants to fix the intrinsic flaw, just like nobody is really interested in fixing the intrinsic memory safety flaws of C. But they shouldn't be denied as real risks, either, or a tradeoff of some kind.
Re:Correctness and composability bugs in the Julia ecosystem by Q6T46nT668w6i3m on Monday May 16, 2022 @09:54AM (#31397225)
@inbounds is a Base feature.
Re:Correctness and composability bugs in the Julia ecosystem by markkitti on Monday May 16, 2022 @10:04AM (#31397389)
Yes, and it is a perfectly fine feature when applied correctly. It would be incorrect to assume that an `AbstractArray` starts at `1` or `0` which is why the updated example now correctly uses `eachindex`: https://docs.julialang.org/en/v1/devdocs/boundscheck/#Elidin...

If you want to assume that an array starts at `1` one needs to require an `Array` rather than an `AbstractArray`.

Re:Correctness and composability bugs in the Julia ecosystem by arksingrad on Monday May 16, 2022 @10:02AM (#31397347)
@inbounds isn't the problem, it's incorrect usage of it. The poor docstring is absolutely a problem though, you should be iterating over eachindex(A), not 1:length(A).