News for hackers. Stuff that matters.

matthewmturner

Re:Apache Arrow DataFusion 7.0 released on Wednesday March 02, 2022 @01:30AM (#30524412)
Highlights from new release

- Significant performance improvements

- Memory tracking and spill to disk

- Support for decimal type

- New datafusion-contrib organization with python implementation and s3 support

- More math functions

- Public expression simplification api

- Arrow2 branch

Currently in the works

- More features for working with limited memory and large datasets

- Ballista improvements on context, state management, extensibility, and file system access