Re:Multiprocessing in Python with Fortran and OpenMP
by milancurcic on Monday November 13, 2017 @04:22PM
(#15690375)
False. OpenMP does shared-memory parallelism and has nothing to do with LAPACK. numpy is completely serial.
[ Parent
| on HN ]
Re:Multiprocessing in Python with Fortran and OpenMP
by Derbasti on Tuesday November 14, 2017 @01:59AM
(#15693258)
Numpy (or rather, BLAS) is running many threads in parallel. The amount of concurrency is actually configurable, and is something well worth optimizing for multiprocessing workloads.
Re:Multiprocessing in Python with Fortran and OpenMP
by auxym on Monday November 13, 2017 @04:38PM
(#15690500)
Numpy can be compiled against a multi-threaded BLAS, for example MKL.
Re:Multiprocessing in Python with Fortran and OpenMP
by santaclaus on Monday November 13, 2017 @07:33PM
(#15691583)
I've seen OpenMP-based, non-BLAS implementations beat out MKL, on Intel hardware nonetheless. We emailed the MKL folks and they just shrugged. :) Don't presume that MKL is the lower limit on performance.
Re:Multiprocessing in Python with Fortran and OpenMP
by pletnes on Monday November 13, 2017 @04:57PM
(#15690644)
Anaconda python is a great option for exactly that. MKL gives great multicore performance. Many packages, e.g scikit-learn, dask and others, lean on numpy for performance, and are thus much faster.
Re:Multiprocessing in Python with Fortran and OpenMP
by milancurcic on Monday November 13, 2017 @05:00PM
(#15690669)
Yes it can, in which case you embed OpenMP parallelism in low-level library calls, and is another way to accomplish what the article suggests.
But it's still a fine-grained parallelism so numpy can match plain Fortran performance in special cases (and not the other way around).