News for hackers. Stuff that matters.

A comment on Python’s Weak Performance Matters. Parent comment

Re:Python’s Weak Performance Matters by fxmc on Wednesday February 07, 2018 @04:53AM (#16323354)
Have you given dask a try? It gives you out-of-core arrays with numpy semantics and distributed computing.
[ Parent | on HN ]
Re:Python’s Weak Performance Matters by VHRanger on Wednesday February 07, 2018 @08:50AM (#16324337)
Dask doesn't solve that problem since it's a wrapper around pandas functions.

If you can't make the core pandas code decently fast, dask won't save you.

[ Parent | on HN ]
Re:Python’s Weak Performance Matters by kiriliponi on Wednesday February 07, 2018 @09:43AM (#16324675)
dask.dataframe might not help but dask.distributed could in that case.

I've had success using it on non vanilla stuff (i.e. code that could not get converted to play natively with numpy/pandas structures)

As a bonus, the nice profiling tools (built within dask) have also helped me improve the performance of the code.

See https://distributed.readthedocs.io/en/latest/

[ Parent | on HN ]
Re:Python’s Weak Performance Matters by pletnes on Wednesday February 07, 2018 @01:20PM (#16326702)
There’s dask.array which works on numpy arrays instead of dataframes. Otherwise, your argument holds.
[ Parent | on HN ]