julia> using LoopVectorization, BenchmarkTools
julia> A = rand(10_000, 10_000); B = rand(10_000, 10_000); C = similar(A);
julia> @benchmark vmapntt!(+, $C, $A, $B)
BenchmarkTools.Trial:
memory estimate: 13.19 KiB
allocs estimate: 91
--------------
minimum time: 29.103 ms (0.00% GC)
median time: 29.202 ms (0.00% GC)
mean time: 29.329 ms (0.00% GC)
maximum time: 30.658 ms (0.00% GC)
--------------
samples: 171
evals/sample: 1
>>> import timeit
>>> u = timeit.Timer("A + B", setup='import numpy as np; A = np.random.rand(10_000, 10_000); B = np.random.rand(10_000,10_000)')
>>> u.repeat(10, 1)
[0.17918606100283796, 0.17888473700440954, 0.17893354399711825, 0.1790916720055975, 0.17922663199715316, 0.17935074500564951, 0.17939399600436445, 0.17940548500337172, 0.17933111900492804, 0.17920518800383434]
Elementwise multiplication is the same fast.
If I get a little more creative and do exp(A) + log(B) instead, multithreaded Julia still takes 30ms, while single threaded slows down to 169ms.
Meanwhile, NumPy slowed down to 847ms, making it 30x slower than multithreaded Julia (on my computer) and 5x slower than single threaded.
Shall I keep making programs more complicated?
I made the arrays huge to get roughly the same ball park of times you reported. I also preallocated the memory in that benchmark to avoid unnecessary allocations (the observed allocations are because multithreading in Julia allocates memory). Is this something as easy to do in Python?
I made a few tweaks to your benchmark code. I get with Julia:
Element-wise sum of two 100x100 matrices (int), (1000 loops)
1.698 ms (2 allocations: 78.20 KiB)
Element-wise multiplication of two 100x100 matrices (float64), (1000 loops)
1.786 ms (2 allocations: 78.20 KiB)
Dot (scalar) product of two 300000 arrays (float64), (1000 loops)
4.801 ms (0 allocations: 0 bytes)
Matrix product of 500x600 and 600x500 matrices (float64)
229.475 μs (0 allocations: 0 bytes)
L2 norm of 500x600 matrix (float64), (1000 loops)
6.566 ms (0 allocations: 0 bytes)
Sort of 500x600 matrix (float64)
523.243 μs (595 allocations: 2.32 MiB)
| Element-wise sum of two 100x100 matrices (int), (1000 loops) | 0.0042680892984208185 |
| Element-wise multiplication of two 100x100 matrices (float64), (1000 loops) | 0.0027007726486772297 |
| Dot (scalar) product of two 300000 arrays (float64), (1000 loops) | 0.0088505959516624 |
| Matrix product of 500x600 and 600x500 matrices (float64) | 0.00107754109922098 |
| L2 norm of 500x600 matrix (float64), (1000 loops) | 0.010691180499998154 |
| Sort of 500x600 matrix (float64) | 0.008244920199649642 |
1. Small int matrix element-wise sum: 2.5x
2. Small float matrix element-wise product: 1.5x
3. Dot product (scalar): 1.84x
4. Matrix multiplication: 4.7x
5. L2 norm: 1.6x
6. Sorting: 15.8x
But I left all the inputs the same.
My primary point with Julia here is that it's easy to optimize it and get more performance out of it if or whenever you need it.
The point of Julia is that Float64 and Int aren't special. They're on the same footing as whatever types you cook up, and our performant multiple dispatch system lets you build a vibrant interacting package ecosystem filled with user defined types doing targeted overloads of other functions.
It's the multiple dispatch and composability of julia code that matters. The important part of the performance of Julia is that we can get the ecosystem benefits of writing pure julia code without paying with runtime performance penalties.
My rule of thumb is that if your cherrypicked microbenchmark involves one specialized Numpy function call, it'll be fast, but if there is two separate specialized Numpy function calls it'll be slow.
Also, it's unlikely that Julia will handily beat the handwritten, carefully optimized C-code in numpy. At that level, it isn't very much about the language but more about the programmers implementing the algorithms. A major selling point of Julia is that you will have fast code even in cases where the code doesn't neatly factor into calling static kernels written in e.g. C.
At this stage I can only see that it is the ecosystem which is hard to evaluate without investing into learning the language. However, I find it hard to believe that it is richer than Java or Python.
Notice how none of those languages have a vibrant numerical computing ecosystem anywhere near Julia's despite being just as old or older than julia and being big, well known languages with large communities? It's almost like scientific programmers don't want to do science in those languages.
I don't think it's an accident that Julia's package developers have been remarkably productive compared to Python. The best differnetial equations library available on Python right now is a wrapper for Julia's DifferentialEquations.jl. This is not because Python users don't care about differential equations or because there's not enough money or manpower. Chris Rackauckas did more on his own in julia for differential equation solving in his first couple years than entire industries did in Python over decades.
Julia is a productive and performant language. When you hold up Java and Python, you're asking me to choose between productivity and performance (at best, obviously Python has serious productivity problems and Java has serious performance problems despite those being their touted strengths).
Well, for that it depends on what you're into. For traditional, coding bootcamp machine learning, Flux.jl is still behind, but I'd point out that the only high performance, usable NeuralODE library is still built on Flux.jl and DifferentialEquations.jl as far as I understand and mostly consists of some import statements and a few function compositions / overloads. Someone already did deep learning on fully homomorphic encrypteded data using Flux.jl. Many 'niche', cutting edge things are still best done with Flux.jl because Flux is fundamentally more flexible than things like PyTorch and TensorFlow. It's using the entire programming language as a framework in a very generic way that PyTorch and Tensorflow can only ape by building ever more complex custom compilers and DSLs with ever diverging semantics from Python.
If Flux.jl is an example of the Julia community's failures to compete I'd say we have very little to worry about, and I look forward to seeing what a well funded commercial entity could build in Julia with a few billion dollars.
When it comes to work, niche languages lose their ground once you start digging deeper and attempt to venture slightly outside their focus domain. They do not have dedicated teams or industrial money and will always lag behind. Look at DL4J, they have a whole company working on it for many years.
Of course time will tell but considering the results so far I am not convinced that investing time into learning a new language will pay off. Especially considering Julia interop with C/C++.
Is the interop bad?
Well, running "Hello, World" still takes 0.3 ms :D
- Hello world is slow? Really? Yeah, it's not a hello-world-oriented programming language.
- One-based indexing is questionable? Troll much?
- Using `ccall` is unsafe? No kidding, there's no way to call a shared library that doesn't crash if you lie to it.
- `@printf` sucks? Then don't use it. I never do. The `print` function works for 99% of cases.
- Julia development is slowing down? Whiffed on that call.
Regarding the C ffi point, you can use Clang.jl to generate correct bindings so that you don't have to read C header files. But note that if you want to call a function in C you need to look at and match its signature, so it's not all that onerous to have to do the same from Julia. Julia's C interop is so good that several other languages have copied it since it came out.
You definitely should have because from what I saw during my google searches this idiotic article was mentioned a couple of times.
Flux is more generic, flexible and thus harder to build. Julia could have been there already if the small ML team w was working with TF type design goals. But they aren't, and therefore are ahead of TF/pytorch in many respects.
Also not as much effort has gone into making this polished for standard deep learning stuff, but you can see the fruit in things like reusing the same code for ML on holomophically encrypted data, faster than pytorch/tf deep learning on graphs and 3d inputs, faster, easier composability with ODEs and prob programming etc.
So any language on Windows able to use COM/UWP, gets those Python libraries, sorry, C++ libraries.
The effort to use Julia FFI to interoperate with them is hardly any different than using Python FFI, with the big difference that in what concerns Julia it would be possible to re-implement them in pure Julia, while the Python community keeps wondering about to write more FFI code to native libraries or finally start adopting PyPy.