News for hackers. Stuff that matters.
Re:The FastLanes File Format [pdf] by westonpace on Thursday July 24, 2025 @06:16PM (#44676976)
Lance contributor here. This sounds about right. We haven't really innovated too much in the compression space. Most of our efforts have been around getting rid of row groups and the resulting changes in decoding patterns.

Our current approach is pretty similar to Parquet for scalar types. We allow a mix of general and lightweight codecs for small types and require lightweight only codecs for larger types (string, binary).

Nice work on the paper :)

There are no comments on this comment yet.