Re:Distributed NumPy Arrays
by canavandl on Friday February 26, 2016 @11:38PM
(#11186086)
There's no JVM overhead like for Spark computation. The dask array methods use the numpy C-API, which are implemented in C and run on the physical machine.
[ Parent
| on HN ]
Re:Distributed NumPy Arrays
by math_and_stuff on Friday February 26, 2016 @11:52PM
(#11186119)
I think you might have misunderstood my comment; I was referring to the bullet point under "What didn't work":
> Reduction speed: The computation of normalized temperature, z, took a surprisingly long time. Iād like to look into what is holding up that computation.