Re:Farewell Pandas, and thanks for all the fish
by 2-3-7-43-1807 on Thursday August 29, 2024 @10:32AM
(#41391209)
even vanilla is faster than dask on a single computer ... _OFFICIALLY_ ... cause dask is and only is for clusters
[ Parent
| on HN ]
Re:Farewell Pandas, and thanks for all the fish
by whimsicalism on Thursday August 29, 2024 @12:11PM
(#41392340)
if you’re at cluster scale, i would skip right over dask and go to spark
Re:Farewell Pandas, and thanks for all the fish
by 2-3-7-43-1807 on Thursday August 29, 2024 @01:04PM
(#41392971)
or hadoop
Re:Farewell Pandas, and thanks for all the fish
by cpcloud on Thursday August 29, 2024 @01:52PM
(#41393488)
I'll also suggest inscribing stone tablets with a chunked version of your computation and distributing the described chunked computation to your friends, and then gathering the tablets and finishing up the compute.
It's scalable to ~billions of nodes.
Re:Farewell Pandas, and thanks for all the fish
by 2-3-7-43-1807 on Thursday August 29, 2024 @02:02PM
(#41393584)
but that exists already and it's called databricks