News for hackers. Stuff that matters.
Re:Apache Arrow Datafusion 5.0.0 release by troelsSteegin on Tuesday August 24, 2021 @02:39PM (#28293054)
Is this the best current view of a roadmap for Datafusion? https://www.mail-archive.com/[email protected]/msg23576.h...
Re:Apache Arrow Datafusion 5.0.0 release by alamb on Tuesday August 24, 2021 @02:54PM (#28293240)
<DataFusion committer here>

I do think that is the best current view of a RoadMap and Vision -- it would be great to flesh it out a bit more.

In fact, I'll make a note to try and add some more higher level context into the project on our goals.

Re:Apache Arrow Datafusion 5.0.0 release by Fordec on Tuesday August 24, 2021 @05:33PM (#28295269)
If the vision is to pseudo-copy the best bits of postgres, I'd be very interested in seeing features that tackle PostGIS type spatial problems. Native spatial work that actually scales to handle global level data in a single node still feels like a pipe dream a lot of the time. Adding things like Dask or xarray feel like hacks on imperfect base layers just to get a base system to be barely operational.