Re:Numba - JIT specializing compiler for annotated Python an...
by takluyver on Sunday June 23, 2013 @01:17PM
(#5928367)
The homepage is a bit dry, but for a look at what Numba is already capable of, this blog post shows that Numba-translated Python code runs about as fast as optimised Cython code (the current tool the SciPy world uses for fast code):
The downside is that Numba can't yet translate any old function you give it, especially if it involves string manipulation (as the name suggests, the focus is numeric). But it's still quite a young tool, and I'm optimistic that that will improve.
Re:Numba - JIT specializing compiler for annotated Python an...
by ninjin on Sunday June 23, 2013 @08:34PM
(#5930231)
As someone that has numerical code at the bottom of my research applications, Numba looks really exciting to me. Shaving off a constant factor means more experiments etc.
But one of the nice things about NumPy is that it allows you to make very complicated operations largely transparent thanks to broadcasting. Just compare pairwise_numpy vs. pairwise_python on the blog post you linked. I don't think anyone really favours the version that you can apply the Numba JIT to.
I know that I will cave eventually when I am desperate for speed, I already do write C99 extensions at times (not for numerical code though). I just wish there was a way to use expressions rather than going full imperative to gain some of that speed.
Re:Numba - JIT specializing compiler for annotated Python an...
by takluyver on Monday June 24, 2013 @02:10PM
(#5934631)
No-one favours the pure Python version at the moment, because we're used to it being unworkably slow. In terms of the actual code, I do prefer pairwise_python in that example, because I can see roughly what's going on. pairwise_numpy is kind of like writing in Perl - it's short, but hard to understand. I use the SciPy stack, and I don't fully understand it. How does slicing a 2D array into three dimensions work?
Re:Numba - JIT specializing compiler for annotated Python an...
by mattip on Sunday June 23, 2013 @03:41PM
(#5929077)
It also needs a way to tell the user what it is doing, pypy has a jit viewer, cython has c code, how do you know what optimizations have been done in numba which AFAIK writes directly to llvm ir?
Re:Numba - JIT specializing compiler for annotated Python an...
by takluyver on Sunday June 23, 2013 @01:39PM
(#5928489)
One area in which Numba probably wins is that it's aware of Numpy arrays as types for the compiled functions. There's a lot of code that handles with Numpy arrays, and Numba integrates nicely with that.
Re:Numba - JIT specializing compiler for annotated Python an...
by takluyver on Sunday June 23, 2013 @04:57PM
(#5929426)
PyPy does, but I don't think that the RPython translation and compilation layer knows about the numpy semantics, so I don't think that helps RPythonic make extension modules.
http://jakevdp.github.io/blog/2013/06/15/numba-vs-cython-tak...
The downside is that Numba can't yet translate any old function you give it, especially if it involves string manipulation (as the name suggests, the focus is numeric). But it's still quite a young tool, and I'm optimistic that that will improve.
But one of the nice things about NumPy is that it allows you to make very complicated operations largely transparent thanks to broadcasting. Just compare pairwise_numpy vs. pairwise_python on the blog post you linked. I don't think anyone really favours the version that you can apply the Numba JIT to.
I know that I will cave eventually when I am desperate for speed, I already do write C99 extensions at times (not for numerical code though). I just wish there was a way to use expressions rather than going full imperative to gain some of that speed.
I'm not sure which approach is better.
Some core PyPy people suggest against using RPython for anything else than what they intended it to use (like writing an interpreter). See:
http://mail.python.org/pipermail/pypy-dev/2013-June/011498.h...
http://mail.python.org/pipermail/pypy-dev/2013-June/011503.h...