News for hackers. Stuff that matters.
Re:Pandas on Ray – Make Pandas faster by chrisaycock on Saturday March 03, 2018 @02:42PM (#16511143)
The one line of code is

  import ray.dataframe as pd
They've replaced many pandas functions with an identical API that runs actions in parallel on top of Ray, a task-parallel library:

https://github.com/ray-project/ray

Unlike Dask, Ray can communicate between processes without serializing and copying data. It uses a shared-memory object store within Apache Arrow:

http://arrow.apache.org/blog/2017/08/08/plasma-in-memory-obj...

Worker processes (scheduled by Ray's computation graph) simply map the required memory region into their address space.

There are no comments on this comment yet.