Easy parallel Python with concurrent.futures

Re:Easy parallel Python with concurrent.futures by dr_zoidberg on Saturday August 15, 2015 @06:00PM (#10067099)

Through Cython you can write extensions that release the GIL to make them easily parellizable. However, given that I mostly work Python in Windows (and 64 bit version), I get that Cython might not be a viable option for your use cases.

As for high performance Python, I always recommend Ian Ozsvalds talks at various conferences. Python can be used to reach high performance, even work well in distributed environments and very paralell architectures. It's owed to the great librarys that were developed to work in those circumstances (Numpy, numba, SciPy, OpenCV[1], to name a few), but it's still something posible within the language. Add that to the rapid development cycle that is posible within it, and the flexibility that it can achieve, and you've got a great language for HPC -- despite it's shortcommings.

[1] I will acknowledge here that OpenCV is not Python-specific, if anyone wishes to point that.

[ Parent | on HN ]

Re:Easy parallel Python with concurrent.futures by eloff on Saturday August 15, 2015 @07:45PM (#10067396)

I'll grant you all of those points, if you use Cython to compile your code to native or use a native library to do the expensive parts of your code, then it works out fine. Both of those get around Python's inherit slowness by doing the costly things outside of Python.

[ Parent | on HN ]