News for hackers. Stuff that matters.
Re:Smaller and faster data compression with Zstandard by Twirrim on Thursday September 01, 2016 @11:26AM (#12406004)
Dug in a bit further. On the non-x86 architecture I use, it looks like it's really just straight core performance that explains it. pigz's only advantage there really seems to be the brute force parallelism.

In particular note the huge difference in branches between gzip and zstd on decompress:

8959780663 branches # 143.024 M/sec

2969481781 branches # 64.454 M/sec

and on misses:

542158823 branch-misses # 6.05% of all branches

89060880 branch-misses # 3.00% of all branches

Re:Smaller and faster data compression with Zstandard by Cyan4973 on Thursday September 01, 2016 @10:20PM (#12410383)
There is now a pzstd implementation, if you wish to compare it to pigz
Re:Smaller and faster data compression with Zstandard by Twirrim on Friday September 02, 2016 @01:54PM (#12414842)
Had a shot. It's slightly buggy, but holy crap is it fast.

I'm not a C programmer, understanding what happened is a bit beyond me but: 1) to compile on linux it needs the -pthread flag passed to it, Makefile is missing that (compiles fine on OS X) 2) decompression over stdin appears to be effectively impossible, still demands in input file. Compression over stdin works fine.