News for hackers. Stuff that matters.
Re:AWS switch from gzip to zstd – about 30% reduction in com... by klauspost on Saturday August 20, 2022 @09:40AM (#32531988)
For MinIO (an S3 compatible server), we add an index for each part, which contains uncompressed -> compressed offset pairs.

Since we already used a Snappy-derived method, each 1MB block is stored without backreferences. With this we only have to decode at most 1MB-1 extra bytes to respond with a specific range offset.

There are no comments on this comment yet.