Zlib-ng: a performance-oriented fork of zlib

Re:Zlib-ng: a performance-oriented fork of zlib by nly on Friday June 05, 2015 @09:12AM (#9665251)

What about decompression? Are we already I/O bound? Compression is irrelevant in many cases, where you can precompress static content.

[ Parent | on HN ]

Re:Zlib-ng: a performance-oriented fork of zlib by moreati on Friday June 05, 2015 @10:03AM (#9665553)

zlib latest takes 3 seconds, zlib-ng takes 13 seconds

    alex@martha:~/src/zlib$ du -sh $TESTFILE
    197M	/home/alex/src/enwiki-20141106-pages-articles-multistream-index.txt.gz
    alex@martha:~/src/zlib$ LD_LIBRARY_PATH=./ time ./minigzip -d <$TESTFILE >/dev/null
    3.36user 0.03system 0:03.87elapsed 87%CPU (0avgtext+0avgdata 1632maxresident)k
    8inputs+0outputs (0major+90minor)pagefaults 0swaps
    alex@martha:~/src/zlib$ LD_LIBRARY_PATH=./ time ./minigzip -d <$TESTFILE >/dev/null
    3.34user 0.04system 0:03.76elapsed 90%CPU (0avgtext+0avgdata 1704maxresident)k
    0inputs+0outputs (0major+91minor)pagefaults 0swaps
    alex@martha:~/src/zlib$ cd ../zlib-ng/
    alex@martha:~/src/zlib-ng$ LD_LIBRARY_PATH=./ time ./minigzip -d <$TESTFILE >/dev/null
    12.20user 0.04system 0:13.63elapsed 89%CPU (0avgtext+0avgdata 1576maxresident)k
    264inputs+0outputs (1major+86minor)pagefaults 0swaps
    alex@martha:~/src/zlib-ng$ LD_LIBRARY_PATH=./ time ./minigzip -d <$TESTFILE >/dev/null
    12.15user 0.08system 0:13.58elapsed 90%CPU (0avgtext+0avgdata 1428maxresident)k
    0inputs+0outputs (0major+84minor)pagefaults 0swaps

[ Parent | on HN ]

Re:Zlib-ng: a performance-oriented fork of zlib by gjm11 on Friday June 05, 2015 @10:33AM (#9665734)

So the optimizations in zlib-ng enable it to decompress ... more than 4x slower than vanilla zlib? That seems worthy of some further investigation.

[ Parent | on HN ]

Re:Zlib-ng: a performance-oriented fork of zlib by pimlottc on Friday June 05, 2015 @01:12PM (#9666636)

I'm curious, was your decompression test file compressed with zlib? zlib-ng? Does it make a difference?

[ Parent | on HN ]

Re:Zlib-ng: a performance-oriented fork of zlib by Dead2 on Friday June 05, 2015 @01:24PM (#9666707)

Unfortunately the minigzip implementation used is using a very minimal codebase to implement the functionality, among these is a problem where it will read and decompress 1 byte per call to inflate(). A patch and pull request for this has been available since yesterday at least, and I merged this today.

In stock zlib this codepath is only used if you defined Z_SOLO, but in zlib-ng it is the default unless you run configure with --zlib-compat (or otherwise set the correct defines).

Please retest with the updated commit, or even better with --zlib-compat if you want to get close to an apples-to-apples comparison.

EDIT: PS: This goes to show that minigzip can not really be used as a reliable benchmark, since actual applications linking to the library could be using it differently (for better or worse).

[ Parent | on HN ]