News for hackers. Stuff that matters.
Re:How macOS manages M1 CPU cores by duskwuff on Monday April 25, 2022 @01:44PM (#31158250)
> it seems like only BZip2 parallelizes nicely

Which is really unfortunate, because bzip2 is not a very good compressor otherwise. It outperforms gzip on compression ratio, but is beat out by anything modern, and is extremely slow, even when parallelized.

A lot of modern compressors can be made parallelizable (and, as a bonus seekable!) by breaking the input/output into chunks and compressing each chunk independently. This loses a bit of compression efficiency, but surprisingly little for sufficiently large blocks.

(This isn't what pigz does, by the way. pigz keeps state between each compressed block, which prevents parallel decoding.)

> For plain old zip this is excusable

Do you mean gzip? PKZIP decompression is actually somewhat parallelizable, because each file is compressed independently of the other files in the archive -- I'm not sure anyone actually does this, though.

Re:How macOS manages M1 CPU cores by jandrese on Monday April 25, 2022 @02:00PM (#31158459)
Last time I looked the information I found online said that due to the way PKZIP files are structured it isn't possible to decompress them in parallel. But checking again it seems that someone had done just that:

https://github.com/jpakkane/parzip

Re:How macOS manages M1 CPU cores by duskwuff on Monday April 25, 2022 @04:19PM (#31160036)
That information was incorrect. Any member in a PKZIP archive can be decompressed independently of any other files in the archive, which is what parzip does.

What you can't do is decompress a single archive member in parallel.

Re:How macOS manages M1 CPU cores by saagarjha on Monday April 25, 2022 @09:54PM (#31163135)
The Xcode XIP is chunked and can be decoded in parallel, but not seekable unfortunately.