I am sad to see that a lot of the zlib-ng seem to be around code style updates, which will make it hard for things to travel back and forth between the codebases. I guess one branch or another will eventually have to "win", which is sad.

Also, is zlib-ng going to commit to keeping the same zlib license? Some of the zlib improvements floating around on the net have snuck in more restrictive licensing. It would be fantastic if zlib-ng could hold the line on license creep.

This is true for many older open source libraries. I wonder:

- Are old platforms still tested as time passes? Is it really worth it maintaining extra cruft for old systems even if it is unknown whether it still works? Are there projects running CI for OS/2 Warp, MS-DOS, or A/UX?

- How many users of old or obscure platforms are there still around? Are they even updating such libraries?

- What platforms do we need to get rid of to get rid of autoconf/automake/configure and use e.g. CMake instead?

Maybe there are tradeoffs or platforms I'm just not aware of, but I at least tend to use C99 as the LCD.

At least VS now supports most of C99. Most. Now about that C11...

C11 would be really nice to see as an LCD, especially for threading (or even just a native POSIX threads implementation).

Why would you want that? Autotools is the devil we know. CMake has its own problems and from a statistical sample of one project implementing both, I came to the conclusion that CMake is more frustrating than Autotools when you need to do something unusual.

Those who do not understand autotools are doomed to reinvent it, poorly.

...and it cannot deal with non-exotic use cases like non-UNIX environments (think Windows + Visual Studio) well at all. In contrast, with CMake you can generate an nmake file or Visual Studio project with one command.

It is for that reason I much prefer autotools, especially at the distribution level where you really don't want to be patching things. CMake can work, but I have had to do more patching for CMake build systems than I have had to for autotools; on that basis I consider autotools "better", at least for our use case.

Unusual in terms of...? If I have learnt anything from Go, Java (per Maven), etc. it's that the far majority of project don't need to do something unusual.

Conversely, if you give people the room to do unusual things, they will.

There's a certain simplicity from the perspective of the user that configure && make && make install works most of the time.

CMake requires a lot of effort to get running by comparison.

zlib latest took 26 seconds to compress 648 MB of text, zlib-ng latest took 16 seconds.

    alex@martha:~/src/zlib$ export TESTFILE=../enwiki-20141106-pages-articles-multistream-index.txt
    alex@martha:~/src/zlib$ du -h $TESTFILE
    648M	/home/alex/src/enwiki-20141106-pages-articles-multistream-index.txt
    alex@martha:~/src/zlib$ LD_LIBRARY_PATH=./ time ./minigzip <$TESTFILE >/dev/null
    23.75user 0.17system 0:26.05elapsed 91%CPU (0avgtext+0avgdata 1864maxresident)k
    56472inputs+0outputs (0major+144minor)pagefaults 0swaps
    alex@martha:~/src/zlib$ LD_LIBRARY_PATH=./ time ./minigzip <$TESTFILE >/dev/null
    24.29user 0.17system 0:26.80elapsed 91%CPU (0avgtext+0avgdata 1772maxresident)k
    0inputs+0outputs (0major+141minor)pagefaults 0swaps
    alex@martha:~/src/zlib$ cd ../zlib-ng
    alex@martha:~/src/zlib-ng$ LD_LIBRARY_PATH=./ time ./minigzip <$TESTFILE >/dev/null
    14.66user 0.11system 0:16.16elapsed 91%CPU (0avgtext+0avgdata 1604maxresident)k
    256inputs+0outputs (0major+140minor)pagefaults 0swaps
    alex@martha:~/src/zlib-ng$ LD_LIBRARY_PATH=./ time ./minigzip <$TESTFILE >/dev/null
    14.64user 0.12system 0:16.09elapsed 91%CPU (0avgtext+0avgdata 1840maxresident)k
    0inputs+0outputs (0major+144minor)pagefaults 0swaps

    alex@martha:~/src/zlib$ du -sh $TESTFILE
    197M	/home/alex/src/enwiki-20141106-pages-articles-multistream-index.txt.gz
    alex@martha:~/src/zlib$ LD_LIBRARY_PATH=./ time ./minigzip -d <$TESTFILE >/dev/null
    3.36user 0.03system 0:03.87elapsed 87%CPU (0avgtext+0avgdata 1632maxresident)k
    8inputs+0outputs (0major+90minor)pagefaults 0swaps
    alex@martha:~/src/zlib$ LD_LIBRARY_PATH=./ time ./minigzip -d <$TESTFILE >/dev/null
    3.34user 0.04system 0:03.76elapsed 90%CPU (0avgtext+0avgdata 1704maxresident)k
    0inputs+0outputs (0major+91minor)pagefaults 0swaps
    alex@martha:~/src/zlib$ cd ../zlib-ng/
    alex@martha:~/src/zlib-ng$ LD_LIBRARY_PATH=./ time ./minigzip -d <$TESTFILE >/dev/null
    12.20user 0.04system 0:13.63elapsed 89%CPU (0avgtext+0avgdata 1576maxresident)k
    264inputs+0outputs (1major+86minor)pagefaults 0swaps
    alex@martha:~/src/zlib-ng$ LD_LIBRARY_PATH=./ time ./minigzip -d <$TESTFILE >/dev/null
    12.15user 0.08system 0:13.58elapsed 90%CPU (0avgtext+0avgdata 1428maxresident)k
    0inputs+0outputs (0major+84minor)pagefaults 0swaps

In stock zlib this codepath is only used if you defined Z_SOLO, but in zlib-ng it is the default unless you run configure with --zlib-compat (or otherwise set the correct defines).

Please retest with the updated commit, or even better with --zlib-compat if you want to get close to an apples-to-apples comparison.

EDIT: PS: This goes to show that minigzip can not really be used as a reliable benchmark, since actual applications linking to the library could be using it differently (for better or worse).

Results with zlib-ng and a newer version of the Cloudflare changes:

https://www.snellman.net/blog/archive/2015-06-05-updated-zli...

Original:

Here are some benchmarks of the Intel and Cloudflare forks that this project is based on:

https://www.snellman.net/blog/archive/2014-08-04-comparison-...

The speedups are not insignificant. (I'll see about updating the post to include results for zlib-ng).

I'm not in the habit of including compression level 6 in the results, since the application I care about defaults to 5 :) And I wanted a good spread of different compression levels in the results.

Also #zlib-ng on freenode if you or anyone else want to discuss.

And my post where I put times and code: http://fizz.buzz/posts/anything-you-zcat-i-zcat-faster-under...

I've seen flabbergasting speedup out of basic CPU-bound utilities simply by recompiling the source code for a fifteen-year-old binary using a modern compiler.

Which also brings up the point that if you want to benchmark zlib-ng, compile zlib with the same compiler.

Github:https://github.com/jtkukunas/zlib