matthewmturner
Re:Apache Arrow DataFusion 7.0 released
on Wednesday March 02, 2022 @01:30AM (#30524412)
Highlights from new release
- Significant performance improvements
- Memory tracking and spill to disk
- Support for decimal type
- New datafusion-contrib organization with python implementation and s3 support
- More math functions
- Public expression simplification api
- Arrow2 branch
Currently in the works
- More features for working with limited memory and large datasets
- Ballista improvements on context, state management, extensibility, and file system access