News for hackers. Stuff that matters.
Re:Zstandard – Fast real-time compression algorithm by glinscott on Sunday November 25, 2018 @10:46PM (#18530892)
If you are interested in compression at all, be sure to take a trip through Charles Bloom's blog [1]. It's an incredible read, he covers everything from the basics all the way through state of the art algorithms.

A great example is this post [2], where he talks about how to correctly implement a Huffman encoder/decoder. It's a lot tricker than it is made to sound in most books. For example, most Huffman codes that are used in practice are length limited, to allow the decoder to use smaller lookup tables. There are a bunch of surprisingly interesting tricks to get that to work well from the encoding side (which symbols do you choose to be smaller than they would be otherwise?).

[1] http://cbloomrants.blogspot.com/ [2] http://cbloomrants.blogspot.com/2010/08/08-12-10-lost-huffma...

Re:Zstandard – Fast real-time compression algorithm by lifthrasiir on Monday November 26, 2018 @04:05AM (#18532036)
I like Charles Bloom's blog, but I'm not sure if it's very approachable (after all, it's "rants" :-). If you want more diversity in the readings, ryg blog [1] makes good reading. Start with the most recent series on efficiently reading bits [2].

[1] https://fgiesen.wordpress.com/category/compression/

[2] https://fgiesen.wordpress.com/2018/02/19/reading-bits-in-far...

Re:Zstandard – Fast real-time compression algorithm by jerryX on Monday November 26, 2018 @03:40AM (#18531958)
Zstandard uses both Huffman and ANS coding: https://en.wikipedia.org/wiki/Asymmetric_numeral_systems
Re:Zstandard – Fast real-time compression algorithm by dman on Monday November 26, 2018 @01:37AM (#18531568)
Thank you for these pointers. Posts like this are why I come to HN :)
Re:Zstandard – Fast real-time compression algorithm by tmd83 on Tuesday November 27, 2018 @06:29AM (#18541412)
What I would really love to know is that how they get those results with their newest set of algorithm. They seem to beat every compression algorithm including zstd and lz4 in both compression and speed. And how we can all benefit from those improvements.