News for hackers. Stuff that matters.
Re:S3 is files, but not a filesystem by cuno on Sunday March 10, 2024 @11:20AM (#39659869)
For example, you can do separate parallel ListObjectV2 for files starting a-f and g-k, etc.. covering the whole key space. You can parallelize recursively based on what is found in the first 1000 entries so that it matches the statistics of the keys. Yes there may be pathological cases, but in practice we find this works very well.

There are no comments on this comment yet.