Re:Improving compression at scale with Zstandard
by terrelln on Thursday December 20, 2018 @09:29PM
(#18731035)
A lot of compression ends up being compress once, decompress one to a few times, so the faster end of the spectrum fits well. Additionally, compression has to fit into the existing system, and existing systems already have tight constraints, so it is an easier sell to say "hey, we can compress faster AND stronger" than what you have right now. For our larger services, we will tune the compression level more carefully, and end up at different places for different services, but still normally around the faster levels.
That said, we still see users of the stronger compression levels.
Re:Improving compression at scale with Zstandard
by JyrkiAlakuijala on Friday December 21, 2018 @09:46PM
(#18738608)
Yes, plenty of use cases -- that's what makes compression so interesting!
In my back-of-the-envelope financial analysis, the highest economic impact is when I can reduce people's need to wait for data to arrive. There we often use relatively slow compression even when it is for one use only. Computers are quite a lot (~1000x) cheaper than people's time.
For example, a mobile client might prefer to have no more than 512 kB sliding window due to design of its memory system or for multi-processing reasons. There, a static resource can still be encoded to the smallest size with extremely slow encoding (quality 9-11) -- but for one-use we'd still prefer a faster encoding (quality 6 or so).
Thank you for explaining your focus. Did you arrive to that by doing economic calculations or was it to have minimal changes to situation with no compression?