Re:Ebiggers/libdeflate: Heavily optimized DEFLATE/zlib/gzip ...
by mxmlnkn on Wednesday August 30, 2023 @02:00PM
(#37326212)
I also did benchmarks with zlib and libarchivemount via their library interface here [0]. It has been a while that I have run them, so I forgot. Unfortunately, I did not add libdeflate. I did not even add ISA-l. At that point, I was already would have been glad if my custom-written gzip decompressor could match the speed of the gzip command line utility, which for some weird reason is half as fast as zlib.
[0] https://github.com/mxmlnkn/rapidgzip/blob/master/src/benchma...
libdeflate-gzip is something I built and installed a while ago from libdeflate/programs/gzip.c. lideflate-gzip -V prints: "gzip compression program v1.18. Copyright 2016 Eric Biggers". I am aware that lots of care also has to be taken with I/O, which might make the command line utility slower than the library interface, but doing the tests in /dev/shm hopefully alleviated this. I am also aware that base64-encoded random data is a weird test case but it has its pros because it is a kind of minimal benchmark for raw Huffman decoding speed without (many) LZ references that need to be resolved.
I redid the benchmark as outlined above with the three test files that I am also using for my extended benchmarks [0]:
File Sizes: Compressed -> Uncompressed: In conclusion, it seems that it highly depends on the test case and the one I tested to, too quickly, check my statement is one of the outliers.[0] https://github.com/mxmlnkn/rapidgzip#scaling-benchmarks-on-2...