Re:Python’s Weak Performance Matters
by kiriliponi on Wednesday February 07, 2018 @09:43AM
(#16324675)
dask.dataframe might not help but dask.distributed could in that case.
I've had success using it on non vanilla stuff (i.e. code that could not get converted to play natively with numpy/pandas structures)
As a bonus, the nice profiling tools (built within dask) have also helped me improve the performance of the code.
If you can't make the core pandas code decently fast, dask won't save you.