News for hackers. Stuff that matters.
Re:Zlib-rs is faster than C by jrockway on Sunday March 16, 2025 @04:47PM (#43382147)
Chromium is kind of stuck with zlib because it's the algorithm that's in the standards, but if you're making your own protocol, you can do even better than this by picking a better algorithm. Zstandard is faster and compresses better. LZ4 is much faster, but not quite as small.

Some reading: https://jolynch.github.io/posts/use_fast_data_algorithms/

(As an aside, at my last job container pushes / pulls were in the development critical path for a lot of workflows. It turns out that sha256 and gzip are responsible for a lot of the time spent during container startup. Fortunately, Zstandard is allowed, and blake3 digests will be allowed soon.)

Re:Zlib-rs is faster than C by jeroenhd on Sunday March 16, 2025 @07:08PM (#43383456)
`Content-Encoding: zstd` was added to Chromium a while ago: https://chromestatus.com/feature/6186023867908096

You can still use deflate for compression, but Brotli and Zstd have been available in all modern browsers for quite some time.

Re:Zlib-rs is faster than C by amaranth on Monday March 17, 2025 @03:26AM (#43385984)
Safari doesn't support zstd, that means if you want to use it you have to support multiple formats.
Re:Zlib-rs is faster than C by jeffbee on Sunday March 16, 2025 @06:36PM (#43383191)
Yeah I just discovered this a few days ago. All the docker-era tools default to gzip but if using, say, bazel rules_oci instead of rules_docker you can turn on zstd for large speedups in push/pull time.
Re:Zlib-rs is faster than C by j16sdiz on Monday March 17, 2025 @02:00AM (#43385563)
Chromium supports brotli and zstd
Re:Zlib-rs is faster than C by cesarb on Monday March 17, 2025 @04:27PM (#43392411)
> Zstandard is faster and compresses better.

However, keep in mind that zstd also needs much more memory. IIRC, it uses by default 8 megabytes as its buffer size (and can be configured to use many times more than that), while zlib uses at most 32 kilobytes, allowing it to run even on small 16-bit processors.