Re:Smaller and faster data compression with Zstandard
by ctur on Wednesday August 31, 2016 @03:15PM
(#12400718)
I'm not sure when videos will be posted -- hopefully soon.
All benchmarks today are single threaded. The algorithm itself is single threaded, but can be parallelized across cores. We will soon release a pzstd command line utility to demonstrate this, similar to pigz, which accelerates both compression and decompression.
Zstandard uses both huff0 and FSE together when it compresses -- it doesn't switch between them based on the input.
Re:Smaller and faster data compression with Zstandard
by twitch_checksum on Monday September 05, 2016 @12:11AM
(#12428051)
Video is now available here :
https://atscaleconference.com/videos/real-time-data-compress...
One thing I haven't figured out from either today's post or Yann's blog is whether Zstandard is switching between huff0 and FSE depending on compression level or is it somehow using both together? Also the post says its both OoO friendly and multi-core friendly but the speed benchmarks are those in a single core context or multi-core? Does only the format/algorithm multi-core friendly or the standard cli can run multi-threaded.