LZAV – Fast In-Memory Data Compression Algorithm (In C)

Re:LZAV – Fast In-Memory Data Compression Algorithm (In C) by duskwuff on Friday July 21, 2023 @02:10AM (#36810541)

Can you give an example of how you're arriving at your performance numbers? My tests with enwik8 show LZAV performing considerably below your benchmarks on an Apple M1 system (~320 MB/s compress, 1780 MB/s decompress), and I'm curious why.

[ Parent | on HN ]

Re:LZAV – Fast In-Memory Data Compression Algorithm (In C) by aleksv on Friday July 21, 2023 @11:40AM (#36815095)

Make sure you are using -O3 compiler option, and are not occasionally compiling into x86_64 code which runs via Rosetta2.

[ Parent | on HN ]

Re:LZAV – Fast In-Memory Data Compression Algorithm (In C) by aleksv on Friday July 21, 2023 @11:53AM (#36815327)

Speeds depend on data as well. Try Silesia dataset first.

[ Parent | on HN ]

Re:LZAV – Fast In-Memory Data Compression Algorithm (In C) by duskwuff on Friday July 21, 2023 @12:43PM (#36816100)

Oh, that'd be it then. I must have overlooked the bit where you mentioned the Silesia dataset and skipped straight to enwik9. With that corrected, I get similar results to yours.

One issue I did notice during testing is that lzav's compression ratio is fairly unimpressive (~42:1) on all-zero data. Surely it should be able to do better than that?

[ Parent | on HN ]

Re:LZAV – Fast In-Memory Data Compression Algorithm (In C) by aleksv on Friday July 21, 2023 @01:02PM (#36816410)

Ah, yes, I'll recheck the constraints regarding such all-zero data, easy to fix.

[ Parent | on HN ]

Re:LZAV – Fast In-Memory Data Compression Algorithm (In C) by aleksv on Saturday July 22, 2023 @11:01PM (#36832201)

You may check lzav 1.4 out, all-zeros runs should compress twice better.

[ Parent | on HN ]