News for hackers. Stuff that matters.

A comment on OpenZL: An Open Source Format-Aware Compression Framework.

Re:OpenZL: An Open Source Format-Aware Compression Framework by d33 on Monday October 06, 2025 @05:10PM (#45496366)
I've recently been wondering: could you re-compress gzip to a better compression format, while keeping all instructions that would let you recover a byte-exact copy of the original file? I often work with huge gzip files and they're a pain to work with, because decompression is slow even with zlib-ng.
[ Parent | on HN ]
Re:OpenZL: An Open Source Format-Aware Compression Framework by o11c on Monday October 06, 2025 @07:51PM (#45497657)
That's called `pristine-gz`, part of the `pristine-tar` project.
[ Parent | on HN ]
Re:OpenZL: An Open Source Format-Aware Compression Framework by d33 on Tuesday October 07, 2025 @06:26AM (#45501384)
Thank you! It seems to be what I'm looking for.
[ Parent | on HN ]
Re:OpenZL: An Open Source Format-Aware Compression Framework by mappu on Monday October 06, 2025 @05:45PM (#45496670)
precomp/antix/... are tools that can bruteforce the original gzip parameters and let you recreate the byte-identical gzip archive.

The output is something like {precomp header}{gzip parameters}{original uncompressed data} which you can then feed to a stronger compressor.

A major use case is if you have a lot of individually gzipped archives with similar internal content, you can precomp them and then use long-range solid compression over all your archives together for massive space savings.

[ Parent | on HN ]
Re:OpenZL: An Open Source Format-Aware Compression Framework by Dylan16807 on Tuesday October 07, 2025 @06:41AM (#45501482)
> A major use case is if you have a lot of individually gzipped archives with similar internal content, you can precomp them and then use long-range solid compression over all your archives together for massive space savings.

Or even a single gzipped archive with similar pieces of content that are more than 32KB apart.

[ Parent | on HN ]
Re:OpenZL: An Open Source Format-Aware Compression Framework by artemisart on Monday October 06, 2025 @05:16PM (#45496425)
I may be misunderstanding the question but that should be just decompressing gzip & compressing with something better like zstd (and saving the gzip options to compress it back), however it won't avoid compressing and decompressing gzip.
[ Parent | on HN ]