S3 is files, but not a filesystem

Re:S3 is files, but not a filesystem by hiyer on Sunday March 10, 2024 @01:25AM (#39657139)

One operation where this difference is significant is renaming a "folder". In UNIX (and even UNIX-y distributed filesystems like HDFS) a rename operation at "folder" level is O(1) as it only involves metadata changes. In S3, renaming a "folder" is O(number of files).

[ Parent | on HN ]

Re:S3 is files, but not a filesystem by okr on Sunday March 10, 2024 @06:19AM (#39658067)

Imho, renaming "folders" on S3 results in copying and deleting O(number of files)

[ Parent | on HN ]

Re:S3 is files, but not a filesystem by hiyer on Sunday March 10, 2024 @09:53PM (#39664171)

Exactly.

[ Parent | on HN ]

Re:S3 is files, but not a filesystem by pepa65 on Sunday March 10, 2024 @06:42AM (#39658161)

From reading the above, if you have a folder 'dir' and a file 'dir/file', after renaming 'dir' to 'folder', you would just have 'folder' and 'dir/file'.

[ Parent | on HN ]

Re:S3 is files, but not a filesystem by klodolph on Sunday March 10, 2024 @11:43AM (#39660054)

There is really no such thing as a folder in S3.

If you have something which is dir/file, then NORMALLY “dir” does not exist at all. Only dir/file exists. There is nothing to rename.

If you happen to have something which is named “dir”, then it’s just another file (a.k.a. object). In that scenario, you have two files (objects) named “dir” and “dir/file”. Weird, but nothing stopping you from doing that. You can also have another object named “dir///../file” or something, although that can be inconvenient, for various reasons.

[ Parent | on HN ]

Re:S3 is files, but not a filesystem by Someone on Sunday March 10, 2024 @07:21AM (#39658307)

> In S3, renaming a "folder" is O(number of files).

More like O(max(number of files, total file size)). You can’t rename objects in S3. To simulate a rename, you have to copy an object and then delete the old one.

Unlike renames in typical file systems, that isn’t atomic (there will be a time period in which both the old and the new object exist), and it becomes slower the larger the file.

[ Parent | on HN ]