News for hackers. Stuff that matters.
Re:Zstandard v1.4.7 by kieckerjan on Thursday December 17, 2020 @12:29PM (#25457784)
The dictionary helps with the boilerplate stuff that is shared between records but not repeated inside them. So it works like a charm for relatively small fragments of structured data like e.g. HTML pages, XML-docs or SQL statements: lots of "SELECT * FROM" fragments that can be replaced by a single token. I suspect it works even better for this kind of data than for natural language data, which tends to be skewed in a Zipf-like way.

There are no comments on this comment yet.