News for hackers. Stuff that matters.

A comment on AWS S3: Sometimes you should press the $100k button. Parent comment

Re:AWS S3: Sometimes you should press the $100k button by base698 on Thursday February 17, 2022 @08:51AM (#30372989)
Yes, and sometimes even listing can take days.

I worked somewhere that a person decided using Twitter Firehose was a good idea for S3. Keyed by tweet per file.

Ended up figuring out a way to get them in batches and condense. Ended up costing about $800 per hour to fix coupled with lifecycle changes they mentioned.

[ Parent | on HN ]
Re:AWS S3: Sometimes you should press the $100k button by orf on Thursday February 17, 2022 @09:44AM (#30373543)
> Yes, and sometimes even listing can take days.

You have a versioned bucket with a lot of delete markers in it. Make sure you've got a lifecycle policy to clean them up.

[ Parent | on HN ]
Re:AWS S3: Sometimes you should press the $100k button by properdine on Thursday February 17, 2022 @12:51PM (#30376253)
Doing an S3 object inventory can be a lifesaver here!
[ Parent | on HN ]