News for hackers. Stuff that matters.
Re:Test for lists in Cython by ZuLuuuuuu on Monday April 19, 2021 @07:26AM (#26861479)
I feel like the #1 downside of Python for the last few years is that you cannot take advantage of multiple cores of a CPU easily. Especially when you think it is heavily used in data analysis. We use Python for data analysis as well, and for 95% of operations we are doing, numpy is fast enough that we don't have any complaints. But sometimes, we do wish to be able to take advantage of all the cores in our CPUs, especially now that we can easily get an 8 core CPU for a reasonable price.

There is multiprocessing module but you cannot share memory between processes. I guess the best options are either writing a C extension or using Numba. Writing C extensions require either distributing binary packages or C compiler to be present on the target computer which is not always ideal. So is using Numba the best solution currently? I tried it a bit in the past but the errors I got was a bit hard to interpret compared to regular Python errors.

I wish the multithreading module had support for native threads. Is there any PEPs trying to bring easy multi-core support for Python?

Re:Test for lists in Cython by szubie on Monday April 19, 2021 @07:31AM (#26861514)
Recent versions of Python (3.8+) have introduced a SharedMemory class for sharing memory between different processes (see https://docs.python.org/3/library/multiprocessing.shared_mem...).

The implementation may still be a bit buggy though, so use with caution! https://bugs.python.org/issue38119

Re:Test for lists in Cython by ZuLuuuuuu on Monday April 19, 2021 @07:42AM (#26861622)
Thank you, I'll definitely check it out!
Re:Test for lists in Cython by anentropic on Monday April 19, 2021 @07:51AM (#26861699)
There are also some libraries built on top of it that might be useful https://github.com/dillonalaird/shared_numpy
Re:Test for lists in Cython by jeremiecoullon on Monday April 19, 2021 @08:02AM (#26861791)
I've been using JAX (https://jax.readthedocs.io/en/latest/) for scientific computing in general (in particular MCMC algorithms), as it's really fast. Even on a CPU you get massive speedups compared to numpy (can be up to 2 or 3 orders of magnitude faster in some cases).

The main selling point of the library is automatic differential and compilation to XLA, but I've been using it even when I don't need gradients, as it's really fast (due to compilation). I also really like the random number generator as it's very good for reproducibility.

I've played around with Julia in the past and really liked it, but in terms speed Jax has pretty much solved that problem for me

Re:Test for lists in Cython by PartiallyTyped on Monday April 19, 2021 @08:09AM (#26861832)
Have you tried numba+numpy? In my experience, it is much faster than Jax and can compile to cuda. It's not caveat free, but it also removes the hustle of labeling arrays as donated in Jax.

You may find this interesting https://github.com/scikit-hep/iminuit/blob/develop/tutorial/...

Re:Test for lists in Cython by cfcf14 on Monday April 19, 2021 @08:19AM (#26861903)
Have you been successful in implementing non-trivial computational code in numba/numpy? I've always found it starts to really break for anything which isn't really trivial, and the errors are mostly non-prescriptive and highly verbose.
Re:Test for lists in Cython by joshuaellinger on Monday April 19, 2021 @09:07AM (#26862313)
I just implemented both a CSV parser and an address standardizer in numba (both CPU and GPU) running in parallel feed through a message queue with a bunch of workers subprocs.

It takes a bit of getting used to but the performance gains on impressive. Basically, my bottlenecks shift from compute to i/o.

I think you have to balance it against writing in C/C++. Mentally, it is basically the same work as writing in C (you manage memory/you write complicated for-loops) but you have good array support with numpy. The primary advantage for me that everything stays in the python runtime environment. You just run the code without any extra steps.

...

What is missing from the timing type 'toy' benchmarks is an understanding that there is typically more than one bottleneck in a real problem and it is easy to choose the wrong one to optimize and get little gains.

After starting C (30 years ago now), spending a long time in C#, then switching to Python a few years ago, I think the unappreciated advantage of python is that I have to abandon all pretense of caring about speed and just get stuff working. It basically solves the pre-mature optimization problem for me by being a fast interpreted language rather than a slow compiled language.

Re:Test for lists in Cython by ZuLuuuuuu on Monday April 19, 2021 @01:40PM (#26865704)
> I think the unappreciated advantage of python is that I have to abandon all pretense of caring about speed and just get stuff working. It basically solves the pre-mature optimization problem for me

I feel the same way. With Python I just write the simplest algorithm that first comes to my mind, even though I know that it is not the most optimized way of doing things. But most of the time I am surprised that it works so fast that I realize I actually don't need to optimize it.

And being able to create and easily manipulate dictionaries and tuples also allows me to create efficient data structures very quickly.

Re:Test for lists in Cython by grej on Monday April 19, 2021 @08:54AM (#26862187)
We have a very large portion of production code written in numba we’ve been running for about 3 years and I made a small contribution to the library. There are a lot of gotchas to numba when the codebase gets large but the benefits far outweigh the downsides. I highly recommend numba.

Edit: also note that a big part of the umap library is written in numba.

Re:Test for lists in Cython by PartiallyTyped on Monday April 19, 2021 @08:33AM (#26862007)
I still have some issues in nojit mode, but jit is most of the time fine. It's still a bit iffy with lists, but most of the time I can use numpy arrays.

TBF I am mainly using it for mostly pure path functions.

Re:Test for lists in Cython by N1H1L on Monday April 19, 2021 @08:39PM (#26869472)
I have a 25,000+ loc scientific codebase in Python, with lot of use in my specialized domain. I gave up on numba, because it's speedups vanish when transitioning to real problems. And secondly, it's bugs are not that intuitive.
Re:Test for lists in Cython by craftinator on Monday April 19, 2021 @08:28AM (#26861966)
I had the same experience about 2 years ago. Maybe it's changed since then? It was nice when you had a pure math function to write, but otherwise seemed to be unreliable, especially in multithreaded and multiprocess situations.
Re:Test for lists in Cython by jeremiecoullon on Monday April 19, 2021 @09:06AM (#26862298)
I haven't tried numba but I've heard good things about it! Nice linked tutorial. If I understand correctly, you pass in jitted functions (using numba, and jax) into iminuit which does the optimisation?

With Jax you can write native for loops that can also be jitted (I imagine you can also do this in numba?); this can then be really fast. Though in that case you would have to write the optimisation algorithm yourself which is not always practical!

Another big speedup in Jax is due to vmap/pmap, which allow to vectorise/parallelise computation. For example you can build a massive gram matrix really quickly using vmap.

Another point: Jax can also run on GPU (like numba :) ) without having to rewrite anything.

Re:Test for lists in Cython by PartiallyTyped on Monday April 19, 2021 @09:28AM (#26862513)
> I haven't tried numba but I've heard good things about it! Nice linked tutorial. If I understand correctly, you pass in jitted functions (using numba, and jax) into iminuit which does the optimisation?

Yes, you can just pass the function and it runs the optimization, in severe cases, you can start by doing a grid `.scan` or `.simplex` (Nelder-Mead simplex method), then `migrad` to minimize, and `.hesse` for 1 sigma bound.

You can also provide a gradient function that, well computes the gradient instead of computing it numerically.

> I imagine you can also do this in numba

Yes! It compiles to native code.

> Another big speedup in Jax is due to vmap/pmap, which allow to vectorise/parallelise computation.

It's possible to compile some vmapped functions with numba too, if the link is any indication, you may see even greater speedup than just jax.grad

However, I do concur that jax's vmap is absolutely fantastic and I found it very useful on many occasions.

Re:Test for lists in Cython by jeremiecoullon on Monday April 19, 2021 @09:36AM (#26862587)
Nice to know that numba has these features; I'll have to check it out at some point!
Re:Test for lists in Cython by pletnes on Monday April 19, 2021 @07:53AM (#26861714)
Numpy operations release the GIL (usually at least) so you can use a threadpool and, indeed, share memory. Just try it and you may be pleasantly surprised.

Dask is great if you’re processing large amounts of data, and it recommends and supports threads for this reason.

Re:Test for lists in Cython by ZuLuuuuuu on Monday April 19, 2021 @09:24AM (#26862460)
I didn't know that. So let's say I have 4 regular Python threads calling the same function, and in this function, let's say I call numpy.add on the same array (but different parts of the array), then will it actually use different cores for these 4 different threads? I will try it out, if it works, then that's actually great and would be super easy.
Re:Test for lists in Cython by pletnes on Monday April 19, 2021 @09:26AM (#26862488)
Great, then my comment was worth it!

Not sure there’s anything to prevent you from foot-shooting if you’re operating on the same array. I’d expect that this is only safe to do if, as you say, arrays are not overlapping. Let me know what you find.

Re:Test for lists in Cython by captain_price7 on Monday April 19, 2021 @06:11PM (#26868471)
The answer is No, isn't it?
Re:Test for lists in Cython by ZuLuuuuuu on Thursday April 22, 2021 @08:02AM (#26901760)
I wrote a small code to test it. Unfortunately it didn't work, but I don't know if the problem is me, maybe I made a mistake...
Re:Test for lists in Cython by yummypaint on Monday April 19, 2021 @09:23PM (#26869757)
What I do is write single-threaded programs that process only a section of data, and manage them with HTCondor. It's some extra overhead and complexity but this way you can easily scale to hundreds of machines without changing anything. You also get mature queuing and job management tools that work independently of your program. If your application is easy to paralellize and you might need more than one machine i highly reccommend this route.
Re:Test for lists in Cython by CogitoCogito on Monday April 19, 2021 @08:10AM (#26861848)
> We use Python for data analysis as well, and for 95% of operations we are doing, numpy is fast enough that we don't have any complaints. But sometimes, we do wish to be able to take advantage of all the cores in our CPUs, especially now that we can easily get an 8 core CPU for a reasonable price.

I understand the sentiment, but in this case I wonder if it is less a python issue and more an issue with numpy. I don't see any reason why numpy couldn't execute the last line of the following multi-threaded:

    >>> import numpy as np
    >>> a = np.arange(1000000000)
    >>> b = 2 \* a
The same thing applies to matrix multiplication. I'm guessing numpy has chosen not to for good reason though.

edit: Actually it seems numpy is multi-threaded under certain circumstances:

https://stackoverflow.com/questions/16617973/why-isnt-numpy-...

Re:Test for lists in Cython by leephillips on Monday April 19, 2021 @09:03AM (#26862277)
Numpy will perform a parallelized calculation when using one of the underlying BLAS routines, if you have the appropriate libraries installed on your machine. So to take advantage of your cores, try to write the calculation as a call to a linear algebra function. Loops or expressions with arithmetic operators as in your example will run on one core. Or, just use Julia.
Re:Test for lists in Cython by N1H1L on Monday April 19, 2021 @08:37PM (#26869459)
Try dask

Distribute your data and run everything as dask.delayed and then compute only at the end.

Also check out legate.numpy from Nvidia which promises to be a drop in numpy replacement that will use all your CPU cores without any tweaks on your part.

https://github.com/nv-legate/legate.numpy

Re:Test for lists in Cython by zozbot234 on Monday April 19, 2021 @07:44AM (#26861636)
> I feel like the #1 downside of Python for the last few years is that you cannot take advantage of multiple cores of a CPU easily.

Also a big downside with JavaScript. Of course both Python and JS are high-level interpreted languages where high-performance use cases aren't the foremost priority.

Re:Test for lists in Cython by azangru on Monday April 19, 2021 @07:52AM (#26861710)
> Also a big downside with JavaScript.

Can't Node run multiple processes (or multiple workers, not sure of the proper terminology) on multiple cores? As discussed in this thread on StackOverflow: https://stackoverflow.com/questions/61893497/node-js-on-mult...

Re:Test for lists in Cython by zozbot234 on Monday April 19, 2021 @07:57AM (#26861760)
> Can't Node run multiple processes

Of course, but you can also run multiple Python processes.

Re:Test for lists in Cython by edgyquant on Monday April 19, 2021 @03:17PM (#26866830)
It’s way different though, node itself is multithreaded (just under the hood)
Re:Test for lists in Cython by willseth on Monday April 19, 2021 @11:56AM (#26864203)
> Of course both Python and JS are high-level interpreted languages where high-performance use cases aren't the foremost priority.

Modern JS engines are all primarily JIT-based, generally only interpreting code when it's faster than waiting on the JIT. JS JITs are quite good now, and in many cases will produce optimized C-equivalent compiled code from fairly naive JS.

Re:Test for lists in Cython by andi999 on Monday April 19, 2021 @07:30AM (#26861505)
Why not use the multiprocessing module? (if you can partition your problem space). I find memory sharing much more difficult to reliably implement an algorithm, and speed gain might be lower through cache conflicts.
Re:Test for lists in Cython by mhh__ on Monday April 19, 2021 @07:35AM (#26861561)
> cache conflicts

This can be an absolute sinkhole for performance, but if you have a large dataset you shouldn't be sharing lines that much one would hope.

Re:Test for lists in Cython by andi999 on Monday April 19, 2021 @07:48AM (#26861675)
Then you shd be able to do process parallelization.
Re:Test for lists in Cython by BlackFly on Monday April 19, 2021 @08:01AM (#26861783)
A real hard requirement is going to be resource pools such as session linked connection pools with associated data. But there are just plenty of places where you don't want to be forced into a message passing architecture for a service and shared memory may be a better fit.
Re:Test for lists in Cython by ekianjo on Monday April 19, 2021 @07:31AM (#26861520)
> Python for the last few years is that you cannot take advantage of multiple cores of a CPU easily.

Same thing in the R ecosystem, it's possible to use multiple cores with the parallel package but there are caveats too.

Re:Test for lists in Cython by clircle on Monday April 19, 2021 @09:49AM (#26862722)
I do as much data work as possible in data.table, which parallelizes things under the hood, and doesn't seem to require me to configure anything.
Re:Test for lists in Cython by curiousgal on Monday April 19, 2021 @07:41AM (#26861607)
I haven't run into any issues with that package honestly (mclapply ftw!). Other than not being able to use it on Windows of course.
Re:Test for lists in Cython by gbrown on Monday April 19, 2021 @08:17AM (#26861894)
It’s process based, so communication overhead is heavy and the infrastructure is minimal - if you’re distributing jobs which are individually large it works fine, but it can’t really be used to replace multithreading in a shared memory model.
Re:Test for lists in Cython by mhh__ on Monday April 19, 2021 @07:30AM (#26861509)
If you are genuinely hitting a wall like that surely it's time to move on from python? Or at least think about it
Re:Test for lists in Cython by ZuLuuuuuu on Monday April 19, 2021 @07:42AM (#26861615)
The thing is that the current ecosystem (numpy + scipy + pandas + PyCharm) fits our company structure perfectly (very few software engineers, mostly test engineers who are not very proficient coders). And we already have tens of thousands lines of code. So changing the whole ecosystem just because 5% of the problems are slow is too big of a jump for us. For now, it is easier for us to write a bit of C code for places where we absolutely need speed. But it would have been so convenient if we could just have native threads, so that we don't even need that 5% C code.
Re:Test for lists in Cython by mhh__ on Monday April 19, 2021 @07:50AM (#26861692)
Do you know what you are actually stalled on? i.e. Memory or Instructions, if it's the latter you can probably eek out some more performance just by fiddling with compiler flags (you'd be surprised simultaneously how clever and how utterly braindead some compilers are if you play with them on Compiler Explorer for a bit)

Also, if you are running on Intel you might have some luck with Intel's profiling tools as, although I've never used them for python they are by far and away the best things in this area, they do support Python.

I don't know what details they give you in from Python code directly, but if you've used perf to profile code at a low level before, vTune is like a Tom & Jerry shotgun in comparison.

Re:Test for lists in Cython by meepmorp on Monday April 19, 2021 @09:51AM (#26862744)
> Compiler Explorer

Never seen this before. Thanks!

Re:Test for lists in Cython by mhh__ on Monday April 19, 2021 @10:01AM (#26862843)
Re:Test for lists in Cython by pantsforbirds on Monday April 19, 2021 @03:28PM (#26866953)
You don't have to replace the core python parts. We moved a decent amount of our data pre-processing into Scala, Golang, and Data Base queries and then just store the processed data for the python models to use to train/run (which ends up using the super efficient C/Fortran/Cuda code).
Re:Test for lists in Cython by killingtime74 on Monday April 19, 2021 @07:36AM (#26861564)
Try Dask