Re:Pigz: Parallel gzip for modern multi-processor, multi-cor...
by gpderetta on Friday May 12, 2023 @10:16AM
(#35916714)
Nice! You should be able to do it without an index by periodically restarting the dictionary on compression and then looking for something resembling the dictionary, right?
Re:Pigz: Parallel gzip for modern multi-processor, multi-cor...
by mgerdts on Friday May 12, 2023 @11:56AM
(#35917988)
Yeah, probably so at the cost of compatibility. As implemented, the .gz file can be used with `gzip -d`.
Re:Pigz: Parallel gzip for modern multi-processor, multi-cor...
by gcflymoto on Friday May 12, 2023 @03:15PM
(#35920889)
Would you mind creating a fork of pigz in GitHub and add this patch? I would be interested in testing it out!
Re:Pigz: Parallel gzip for modern multi-processor, multi-cor...
by mgerdts on Friday May 12, 2023 @03:56PM
(#35921479)
You can grab the version from the solaris userland repo I linked and use it without me completing a homework assignment. Just grab the pigz-2.3.4 source then apply the patches from [1] in the proper order. Maybe some of them aren't needed for non-Solaris.
1. https://github.com/oracle/solaris-userland/tree/master/compo...
I thought I had opened a PR for that a long while ago, but it doesn't show up on github these days. In any case, I did ask Mark Adler to review it. It was never a priority, then the code changed in ways that I don't really want to deal with.
While looking through the PRs, I noticed a PR for Blocked GZip Format (BGZF) [2]. That's very interesting, and perhaps suggests that bgzip is a tool you would be interested in.
https://github.com/oracle/solaris-userland/blob/master/compo...
Shortly after submitting a PR the code went through major surgery, and my patch then needed a similar amount of surgery. Oracle then whacked most of the Solaris org, and I don’t think this ever got updated to work with the current pigz.