Behind Amazon’s HQ2 fiasco

Re:Behind Amazon’s HQ2 fiasco by tomc1985 on Tuesday February 04, 2020 @02:35PM (#22239982)

As in, search results reflect only the current visible page and not all pages. If you use a bucket as a dump for, say, daily backups, finding a specific file from 3 years ago is really time-consuming and needlessly difficult. Can't sort, can't search.... just pure frustration

I haven't checked this feature in years -- maybe they fixed it

[ Parent | on HN ]

Re:Behind Amazon’s HQ2 fiasco by scarface74 on Tuesday February 04, 2020 @05:57PM (#22242037)

It comes from a fundamental lack of understanding of what S3 is and how it is organized. Objects are indexed as a key value store optimized for sorting on a key and a key prefix. The fact that keys can have backslashes as a character doesn’t mean that objects are stored in “folders”. Knowing how S3 indexes data - by key only and all other metadata is tagged on the object and not in the index, should inform how to name your keys and your access patterns.

For instance sorting by date, would involve AWS querying each objects meta data and then sorting it.

If having to find objects by date is an important use case, why not create keys in the format of YYYYMMDD?

Or just use the CLI.

https://github.com/aws/aws-cli/issues/2095

[ Parent | on HN ]