News for hackers. Stuff that matters.
Re:Improving compression at scale with Zstandard by felixhandte on Wednesday December 19, 2018 @07:17PM (#18721330)
The Visual Display of Quantitative Information is one of my favorite books!

0 in this case is not really a relevant value (since that would mean transforming the input into something infinitely large). The functional identity value / origin here is 1x. Here's what that looks like [1].

To me, this is a significantly less useful image. But maybe that stems from a lot of comfort with both the subject matter and the detail log-log plots that I work with to evaluate zstd performance, e.g. [2].

[1] https://imgur.com/gU2Gdf6 [2] https://github.com/facebook/zstd/pull/1317#issuecomment-4260...

Re:Improving compression at scale with Zstandard by hinkley on Wednesday December 19, 2018 @07:53PM (#18721558)
Hey thanks for the new chart, and you're right about 1x. But we might have to disagree because I don't think this graph is that bad. I can see a bunch of talking points in this graph that are about strategy instead of about why I hate bad graphs.

First, that data point for lz4 out around 820 MB/s kind of throws the groove off of that graph. Despite that, I would still suggest you prune the graph off at 850 or 900 to reduce the squish of the horizontal data (pruning the graph to put the last value at 100% of width or height isn't considered a no-no).

One of the things I can see now but couldn't before is that the size/speed tradeoff is pretty linear except for the giant dog-leg around 3.6:1, and a less pronounced but still notable one occurs at 2.9:1.

If I sat down with my team to discuss this chart I'd suggest we agree that we aren't interested in anything above 3.6:1. And then I'd suggest we look at everything above 2.9:1, but my eye is on 3.2:1 (where zlib tops out and is 1/10th the speed). If they don't like 2.9:1, then the next interesting point is at about 2.7:1 when zlib bottoms out. We're already at such a high bandwidth rate that something else is probably going to be the bottleneck.