News for hackers. Stuff that matters.
Re:S3 is files, but not a filesystem by paulddraper on Sunday March 10, 2024 @09:35PM (#39664052)
> Recursively listing these files

There's no "recursive" nature to S3 buckets. "Listing a directory" is simply listing keys by a prefix.

So list by the upper-most prefix that you want. If you have 1,000,000 files, it will take 1,000 API calls to list everything.

If each call takes 1s (I have no idea what your latency to the S3 bucket region is), then it will indeed take 15 min.

https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObje...

There are no comments on this comment yet.