Re:Swift for TensorFlow Shuts Down
by freedomben on Friday February 12, 2021 @06:56PM
(#26120057)
Julia is fascinating to me, and I don't do any ML (currently). Julia seems like such a well designed language. I'm excited to see where it goes.
As weird as it sounds, the only thing that stops me from trying Julia is 1 indexing
Re:Swift for TensorFlow Shuts Down
by adgjlsfhk1 on Saturday February 13, 2021 @02:44AM
(#26122088)
IMO, Julia is good enough that you might not notice. you can use constructions like `for i in axes(V)` that will automatically give you the indexes, or `V[begin:begin+2]` to get the first 2 elements. Also higher order functions eg. `map` (and broadcasting) mean that you will rarely be manually working with indexes.
You still need to access indexes if you're working with arrays though, which is a very common data structure.
Re:Swift for TensorFlow Shuts Down
by improbable22 on Saturday February 13, 2021 @07:25AM
(#26123216)
That's not so obvious, the abstractions for multi-dimensional indexing are better than (I think) you're imagining.
Here's a nice recent example: a new reverse!(A; dims) function, which is fast and works on arrays with any number of dimensions. And is completely indifferent to where the indices start:
https://github.com/JuliaLang/julia/pull/37367/files#diff-eb9...
I'm not 100% sure what I'm looking at, but that seems to be related to reversing arrays/matrices, not accessing arrays through indexes.
Re:Swift for TensorFlow Shuts Down
by improbable22 on Saturday February 13, 2021 @07:59AM
(#26123353)
Maybe less readable than I remembered, sorry! But these `CartesianIndex` things are how you index multi-dimensional arrays.
If by indexing you only mean adding offsets to pointers with your bare hands, this is possible but discouraged. Because you'll produce a buggier and less general version of exactly what already exists. In this case, reversing an N-dimensional array, along any M of its axes, with arbitrary offsets -- there are a lot of ways to get that wrong.
What I mean is something like this:
# one dimensional
a = [1, 2, 3]
# prints the number 1 @ index 0
print(a[0])
# multi-dimensional
b = [[1, 2, 3], [4, 5, 6]]
# prints the number 4 @ index (1, 0)
print(b[1][0])
Re:Swift for TensorFlow Shuts Down
by improbable22 on Saturday February 13, 2021 @08:59AM
(#26123617)
OK, I thought you were advocating zero based on complicated index calculations for some algorithm, with mod & div, which is what I was trying to say could be abstracted away. The literal syntax `a=[1,2,3]` makes a one-dimensional `Array{Int,1}` whose first element is indeed `a[1]`. (`b` is an array of arrays, while `c = [1 2 3; 4 5 6]` is an honest 2-dimensional array.)
Because you believe that you can't get used to a different convention? It gets brought up a lot that for array offsets, zero based is more natural. In my experience, the amount of index calculations you need in Julia is minimal anyway because there are really nice abstractions for n-dimensional indexing.
Re:Swift for TensorFlow Shuts Down
by viktorcode on Saturday February 13, 2021 @04:30AM
(#26122500)
0-based indexing is an artefact of pointer math. Modern languages don't treat arrays as a pointers to their first element. So, it is still with us only because almost no language designer decided to challenge existing convention.
Re:Swift for TensorFlow Shuts Down
by threatofrain on Saturday February 13, 2021 @06:36AM
(#26122998)
I see 1-indexing as an artifact of a lot of linear algebra texts, whereas 0-indexing is more popular in analysis. Julia just wants it to be easier to directly copy formulas from publications.
I also think it’s blurry as to whether mathematicians and programmers see indexing numbers as merely indices or richer numbers.
The beautiful thing of having polyglot experience, since the days when C was just another language trying to get users, is not being put down by minutia like that.
Just a reminder that latency has been aggressively worked on in 1.5 and 1.6 releases. Example graph. Https://i.redd.it/ik4uymvb28k51.png
Re:Swift for TensorFlow Shuts Down
by gsathya_hn on Saturday February 13, 2021 @09:56AM
(#26123903)
> Julia uses a method JIT, which are quick to implement, but which makes latency gets bad. Meaning if you load a whole new package, running the first function can cause some delay. With a JavaScript style tracer JIT added into the mix one could have probably managed to have both high performance and low latency.
This is actually the other way around -- Method JITs are harder to implement and all the major JavaScript JITs are method JITs, not tracing JITs. There are many reasons that contribute to the latency but I don't see what makes method JITs inherently slower than tracing JITs.
I like Julia a lot, and I think it's the best option for ML research, but isn't there a big advantage to Swift or similar languages when it comes to production deployments on mobiles? The distribution of compiled binaries in Julia seems a bit kludgy and less reliable than with a statically compiled language like Swift.
Doing machine learning stuff in Julia is simply much more user friendly than doing the same in Swift. Swift is nice for iOS development, but I think in data science and Machine Learning Julia will always be a much better choice.
It is a pity Google did not go for Julia. Julia has gotten exceptionally far despite limited resources. With Google style resources, Julia could have been massive.
Despite modest investment, the Julia JIT beats almost anything out there. It would have been amazing to see what they could have pulled off if they had put the kind of resources that was put into the JavaScript V8 JIT into the Julia JIT.
Julia uses a method JIT, which are quick to implement, but which makes latency gets bad. Meaning if you load a whole new package, running the first function can cause some delay. With a JavaScript style tracer JIT added into the mix one could have probably managed to have both high performance and low latency.
Alternatively with more investment they could have had better precompile caching, meaning libraries that had been loaded in the past would JIT really fast. Julia does that today, but it could have worked a lot better than it currently does. It isn't that it can't be done, but it just requires resources.