News for hackers. Stuff that matters.
Re:MinLZ: Efficient and fast Snappy/LZ4 style compressor in ... by klauspost on Monday March 17, 2025 @04:01PM (#43392208)
Re:MinLZ: Efficient and fast Snappy/LZ4 style compressor in ... by ptman on Tuesday March 18, 2025 @04:06AM (#43396781)
The blog post is quite nice: https://blog.min.io/minlz-compression-algorithm/

But comparisons with zstd & lz4 would be nice.

Is someone planning on a wuffs/rust implementation with C API so that it wouldn't be Go only?

Re:MinLZ: Efficient and fast Snappy/LZ4 style compressor in ... by klauspost on Tuesday March 18, 2025 @07:56AM (#43398340)
The main reason I took the effort to write the spec was to encourage ports - and commit to the format.

I've converted a basic block encoder/decoder to C (see end of README for link), to assist anyone wanting to get started on a C implementation.

Having been 10+ years since I've done any serious C, I didn't want to assume I could write a good C API for it.

If anyone are interested in picking it up, reach out.

> comparisons with zstd & lz4

The repo has lz4 comparisons (Go version) for both blocks and streams.

I see MinLZ more like a compliment to zstd, just like lz4 - where speed matter more than ultimate size.

Here is the fully parallel speed of block decompressing with zstd or minlz - Go versions:

* Protobuf - zstd: 31,597 MB/s - MinLZ 155,804 MB/s. * HTML - zstd: 25,157 MB/s - MinLZ 82,292 MB/s. * URL List - zstd: 16,869 MB/s - MinLZ 45,521 MB/s. * GEO data - zstd: 11,837 MB/s - MinLZ 36,566 MB/s.

Of course zstd compresses to a smaller size - but for things like streams transfers or for fast readbacks from disk you probably want the fastest.

Overall, I would say that MinLZ is about 2x faster decompressing, and compresses about 2x the speed compressing at max speed. But of course with less compression.