News for hackers. Stuff that matters.

A comment on AWS switch from gzip to zstd – about 30% reduction in compressed S3 storage. Parent comment

Re:AWS switch from gzip to zstd – about 30% reduction in com... by alexchamberlain on Saturday August 20, 2022 @04:09AM (#32530249)
+1 if you are storing objects uncompressed, I'd be amazed if AWS doesn't compress them and charge you for the full space anyway
[ Parent | on HN ]
Re:AWS switch from gzip to zstd – about 30% reduction in com... by sitkack on Saturday August 20, 2022 @04:45AM (#32530458)
If this true, there is possibly a side channel one could run against object storage to determine if someone else in the content-addressable-store has the same files.

Like when it was easy to file share on dropbox by having the correct hashes. A GUID could summon a 1GB file.

[ Parent | on HN ]
Re:AWS switch from gzip to zstd – about 30% reduction in com... by eurg on Saturday August 20, 2022 @05:02AM (#32530549)
Compression and content-addressing are two separate things.

Content addressing across accounts on private, AWS encrypted S3 buckets would run counter to their claims.

[ Parent | on HN ]
Re:AWS switch from gzip to zstd – about 30% reduction in com... by alexchamberlain on Saturday August 20, 2022 @05:36AM (#32530711)
A couple of comments across the thread have made similar points, but if I were implementing this, the "client metadata" like the incoming sha256 etc would be implemented a layer higher than the actual byte storage, so the byte storage could be compressed without any impact on that sort of thing.
[ Parent | on HN ]
Re:AWS switch from gzip to zstd – about 30% reduction in com... by staticassertion on Saturday August 20, 2022 @05:01AM (#32530539)
That assumes cross-tenant compression.
[ Parent | on HN ]