Re:Dask – A flexible library for parallel computing in Python
by mianos on Thursday November 18, 2021 @05:51AM
(#29264090)
Funny and I had a similar real life experience with a hadoop cluster. I replaced a job that needed 10 machines with a few lines of C, some python and GNU parallel, which was perl I believe at the time. The process was taking over 24 hours on the cluster and became unworkable. I got it back to 2 hours on my slow Macbook air.
On Dask, it may not be super fast for very large data sets but it is sure easy to use.
On Dask, it may not be super fast for very large data sets but it is sure easy to use.