News for hackers. Stuff that matters.

A comment on AWS switch from gzip to zstd – about 30% reduction in compressed S3 storage. Parent comment

Re:AWS switch from gzip to zstd – about 30% reduction in com... by IntelMiner on Saturday August 20, 2022 @05:34AM (#32530696)
Presumably it's the tradeoff of CPU overhead versus disk and bandwidth (larger files take longer to copy into memory, which is also energy usage. And more bandwidth to shunt around Amazon's own network)
[ Parent | on HN ]
Re:AWS switch from gzip to zstd – about 30% reduction in com... by SuperQue on Saturday August 20, 2022 @09:03AM (#32531771)
There's also a latency component.

Since CPUs are fast enough to deflate in real-time now, your bottleneck for a read is your storage/network.

Reducing the bytes read from storage improves the IO latency.

[ Parent | on HN ]