Re:The FastLanes File Format [pdf]
by nly on Thursday July 24, 2025 @02:11PM
(#44674076)
My experience with parquet is that the (C++) libraries are pretty awful and resulting performance lackluster
[ Parent
| on HN ]
Re:The FastLanes File Format [pdf]
by azimafroozeh on Thursday July 24, 2025 @02:56PM
(#44674643)
I totally agree — the C++ implementations of Parquet aren’t the best they could be. To be fair, though, it’s a tough problem: Parquet supports a broad range of encodings and compression schemes, and to do that, it pulls in a lot of dependencies and requires complex dependency management.
That’s one of the main reasons we built FastLanes from scratch instead of trying to integrate with Parquet.
With FastLanes, we’ve taken a different approach: zero dependencies, no SIMD intrinsics, and a design that’s fully auto-vectorizable. The result is simpler code that still delivers high performance.