Apache Arrow DataFusion 7.0 released Posted by matthewmturner on Wednesday March 02, 2022 @01:30AM from the arrow.apache.org dept. https://arrow.apache.org/blog/2022/02/28/datafusion-7.0.0/ ( 2 comments | 12 points | on HN )
- Significant performance improvements
- Memory tracking and spill to disk
- Support for decimal type
- New datafusion-contrib organization with python implementation and s3 support
- More math functions
- Public expression simplification api
- Arrow2 branch
Currently in the works
- More features for working with limited memory and large datasets
- Ballista improvements on context, state management, extensibility, and file system access