hashjoiner
Re:Polars DataFrame Processing on GPU's
on Thursday April 04, 2024 @03:59PM (#39935112)
Very interesting development :)
Re:Apache Arrow Datafusion 5.0.0 release
on Wednesday August 25, 2021 @04:03AM (#28299097)
DataFusion committee here.
DuckDB can work on Arrow data, so I think it could coexist with DataFusion / Ballista quite well. I am not sure whether having different execution engines for Ballista is on the roadmap, but it's certainly a possibility!
Re:Apache Arrow Datafusion 5.0.0 release
on Wednesday August 25, 2021 @04:00AM (#28299073)
Yes, that's pretty exciting!
There is even support in Polars to execute the dataframe compute in DataFusion (as it can handle larger than memory datasets).
Re:Apache Arrow Datafusion 5.0.0 release
on Tuesday August 24, 2021 @03:22PM (#28293598)
There is a PR from me (Daniël, committer) with for db-benchmark. For the group by benchmarks, on my machine, it is currently somewhat slower than the fastest (Polars).
https://github.com/h2oai/db-benchmark/pull/182
Also we do support running TPC-H benchmarks. For the queries we can run, those are already finishing faster than Spark. We are planning to do more benchmarking and optimizations in the future.
Re:Datafusion 5.0.0 release with major new features and performance im...
on Friday August 20, 2021 @04:56PM (#28251316)
Happy for this release, and excited to contribute to this great query engine.