News for hackers. Stuff that matters.

A comment on Pigz: Parallel gzip for modern multi-processor, multi-core machines. Parent comment

Re:Pigz: Parallel gzip for modern multi-processor, multi-cor... by aigoochamna on Friday May 12, 2023 @08:33AM (#35915289)
How big is that file... I have 2TB files compressed down to ~300GB and gunzip'ing them takes ~2-3 hours. Granted, that's still a long ass time, but not 2-3 days.

If anything, I wonder what kind of hard drive John has. If you're reading them off a network drive backed by tape drums it's probably going to take a while ;P

[ Parent | on HN ]
Re:Pigz: Parallel gzip for modern multi-processor, multi-cor... by lathiat on Friday May 12, 2023 @08:38AM (#35915351)
bzip2 is much slower than gzip.
[ Parent | on HN ]
Re:Pigz: Parallel gzip for modern multi-processor, multi-cor... by mobilio on Friday May 12, 2023 @09:18AM (#35915826)
Very bzip2 problem is that even decompression is slower.
[ Parent | on HN ]
Re:Pigz: Parallel gzip for modern multi-processor, multi-cor... by aidenn0 on Friday May 12, 2023 @03:12PM (#35920833)
bzip2 is not much slower at compression than gzip; gzip is far faster at decompression though. Either way since bzip2 is a block based compressor, parallelization is trivial, and parallel implementations started appearing about 20 years ago; pbzip2 is almost certainly in whichever package manager is in use for TFA.
[ Parent | on HN ]