News for hackers. Stuff that matters.
Re:DuckDB performance improvements with the latest release by iwd on Monday November 06, 2023 @01:31PM (#38166656)
Do you have compression enabled? At least from Pandas, Parquet defaults to compressed and Arrow/Feather default to uncompressed. When I enable zstd compression, I get similar file sizes, and sometimes Arrow is smaller.
Re:DuckDB performance improvements with the latest release by slt2021 on Monday November 06, 2023 @01:39PM (#38166785)
I was just trying pandas native .to_parquet and .to_arrow() without any extra config knobs