Show HN: Zeekstd – Rust Implementation of the ZSTD Seekable Format

Re:Show HN: Zeekstd – Rust Implementation of the ZSTD Seekab... by threeducks on Monday June 16, 2025 @11:53AM (#44290762)

Assuming that frames come at a cost, how much larger are the seekable zstd files? Perhaps as a graph based on frame size and for different kinds of data (text, binaries, ...).

[ Parent | on HN ]

Re:Show HN: Zeekstd – Rust Implementation of the ZSTD Seekab... by mcraiha on Monday June 16, 2025 @12:18PM (#44290972)

It depends on content and compression options. ZSTD has four different compression methods: Raw literals, RLE literals, Compressed literals and Treeless literals. I assume that the last two might suffer the most if content is splitted.

[ Parent | on HN ]

Re:Show HN: Zeekstd – Rust Implementation of the ZSTD Seekab... by rorosen on Tuesday June 17, 2025 @02:21AM (#44296197)

It depends on the frame size you choose. Every frame requires a few bytes of additional metadata, how much exactly depends on other compression settings (e.g. frame checksums, which are 4 byte, are only present if enabled). I just tested with a 1G file and compression level 3. zstd compresses it to 559M, zeekstd with a 2M frame to 565M. If I increase the frame size to 4M, zeekstd yields 562M.

I will add a section to the readme, this is a good question that other people might have too!

[ Parent | on HN ]