News for hackers. Stuff that matters.

A comment on New standards for a faster and more private Internet.

Re:New standards for a faster and more private Internet by lifthrasiir on Thursday September 26, 2024 @08:31PM (#41665060)
The benchmark for Zstandard against Brotli seems to miss a key information---the compression levels used for both algorithms, because both the compression ratio and compression time will depend on them. In fact this had been my long suspicion about introducing Zstandard to the web standard, because lower compression levels for Brotli are not that slow and it was never publicly mentioned whether improving lower Brotli levels deemed infeasible or not. Given Zstandard Content-Encoding was initially proposed by Meta, I'm not even sure they have at least tried.

Given we now have two strictly better algorithms than gzip, I also wonder about a hybrid scheme that starts with Zstandard but switches to Brotli when the compression time is no longer significant for given request. We might even be able to cheaply convert the existing Zstandard stream into Brotli with some restrictions, as they are really LZSS behind the scene?

[ Parent | on HN ]
Re:New standards for a faster and more private Internet by terrelln on Thursday September 26, 2024 @09:17PM (#41665337)
Meta drove the Zstandard content encoding, but Google drove the adoption of Zstandard in Chrome.

The faster Brotli levels could probably be made to match Zstandard’s compression speed. But we’ve invested a lot in optimizing these levels, so it would likely take significant investment to match. Google is also contributing to improving the speed of Zstandard compression.

A cheaper conversion from Zstandard to Brotli is possible, but I wouldn’t really expect an improvement to compressed size. The encoding scheme impacts how efficient a LZ coding is, so for Brotli to beat Zstandard, it would likely want a different LZ than Zstandard uses. The same applies for a conversion from Brotli to Zstandard.

[ Parent | on HN ]
Re:New standards for a faster and more private Internet by lifthrasiir on Thursday September 26, 2024 @09:48PM (#41665484)
The conversion is meant to avoid the expensive backreference optimization when it has been already done once by Zstandard, because you can't prepend a Zstandard bitstream with a Brotli bitstream without turning one to another. But well, I think such hybrid scheme is hard to make when the latency is at stake.
[ Parent | on HN ]