S3 is files, but not a filesystem

Re:S3 is files, but not a filesystem by mickael-kerjean on Sunday March 10, 2024 @01:20AM (#39657117)

Fun experiment I made with my mum, building a storage independent dropbox like UI [1] for anything that implement this interface:

  type IBackend interface {
    Ls(path string) ([]os.FileInfo, error)
    Cat(path string) (io.ReadCloser, error)
    Mkdir(path string) error
    Rm(path string) error
    Mv(from string, to string) error
    Save(path string, file io.Reader) error
    Touch(path string) error
  }

My mum really couldn't care less about the posix semantic as soon as she can see the pictures of my kid which happen to be on S3

[1] https://github.com/mickael-kerjean/filestash

[ Parent | on HN ]

Re:S3 is files, but not a filesystem by wwalexander on Sunday March 10, 2024 @01:28AM (#39657148)

Reducing things to basically the interface you laid out is the point of 9p [1], and is what Plan 9’s UNIX-but-distributed design was built on top of. Same inventor as Go! If you haven’t dived down the Plan 9 rabbit hole yet, it’s a beautiful and haunting vision of how simple cloud computing could have been.

[1] https://9fans.github.io/plan9port/man/man9/intro.html

[ Parent | on HN ]

Re:S3 is files, but not a filesystem by MrJohz on Sunday March 10, 2024 @08:38AM (#39658600)

I think this interface is less interesting than the semantics behind it, particularly when it comes to concurrency: what happens when you delete a folder, and then try and create a file in that folder at the same time? What happens when you move a folder to a new location, and during that move, delete the new or old folders?

Like yes, for your mum's use case, with a single user, it's probably not all that important that you cover those edge cases, but every time I've built pseudo-filesystems on top of non-filesystem storage APIs, those sorts of semantic questions have been where all the problems have hidden. It's not particularly hard to implement the interface you've described, but it's very hard to do it in such a way that, for example, you never have dangling files that exist but aren't contained in any folder, or that you never have multiple files with the same path, and so on.

[ Parent | on HN ]

Re:S3 is files, but not a filesystem by mickael-kerjean on Sunday March 10, 2024 @07:22PM (#39663328)

All those considerations are important when implementing the interface but the interface itself isn't invalidated by those concerns and can cope with those constraints fine.

[ Parent | on HN ]