Re:Show HN: Reimagine the File System
by lapinot on Monday November 30, 2020 @05:16PM
(#25259737)
Neat. As a remark, this is how LDAP is structured: entries (files) are "tagged" by attributes (directories) and you access them with the same pseudo-paths which are actually conjunctions of filters.
I must say honestly i'm not so fond of using ordered sequences (posix paths) in place of the unordered sets which they should be. I guess my question is really why you wanted to make a fuse fs out of it (versus say some tag-based filtering script like notmuch). (just for fun is surely a valid answer)
Re:Show HN: Reimagine the File System
by daenz on Monday November 30, 2020 @10:03PM
(#25261817)
Basically what Spivak said. I was interested in adapting an existing interface (a filesystem) to do the set intersections because it means more software could benefit from it without additional work.
Re:Show HN: Reimagine the File System
by eyelidlessness on Monday November 30, 2020 @04:58PM
(#25259569)
I can’t speak to the implementation but in principle, a filesystem essentially being a specialized database, there’s no reason techniques commonly used for SQL optimization couldn’t be applied here. It’s fundamentally set theory, where you’re obtaining your results by producing subsets rather than traversing a hierarchical tree of branches and end nodes. Of course that implies some pathological conditions (some superset/subset scenarios will produce too many results to be useful or to optimize resolution) which would produce poor UX, but sane defaults and heuristics could alleviate that.
Again not speaking to the project itself, but in principle achieving good performance on something like this would boil down to “do what SQL databases do, then design efficient schemas and queries for the use case”.
Re:Show HN: Reimagine the File System
by daenz on Monday November 30, 2020 @03:33PM
(#25258596)
Not formally, no, but anecdotally, I do notice some performance impact with with many files and tags, and some of the more aggressive file browsers like to depth-first cache directories, which can be involved when directories are generated dynamically and extremely deep. I am actively looking for solutions to that!
Fortunately you can try Supertag out without touching your original files, as Supertag will use symlinks to your original files, so you don't need to move them.
- Library curation (i.e. individual or mass categorization)
- Content discovery/rediscovery (i.e. find things based on metadata rather than location)
It’ll either poorly serve or need additional consideration for:
- Contained dependencies (i.e. any kind of encapsulated package that has internal references)
- Structural authorization (i.e. a given user may or may not be permitted access to some or all functions based on the branch/leaf they’re trying to access)
- Structural layout (i.e. the user has a hierarchy in mind which they wish to reify)
Re:Show HN: Reimagine the File System
by eyelidlessness on Monday November 30, 2020 @05:16PM
(#25259741)
I’ll leave the above comment intact from here but I already edited once and still regret the word choice “fundamentally incompatible”. As with any data structure, you can make adaptations and accommodations to support use cases that seem to conflict. There’s always a memory/performance/impedance mismatch/experience cost for any abstraction, but there’s no reason a set/tree hybrid couldn’t provide for both families of use cases, given those costs.
I must say honestly i'm not so fond of using ordered sequences (posix paths) in place of the unordered sets which they should be. I guess my question is really why you wanted to make a fuse fs out of it (versus say some tag-based filtering script like notmuch). (just for fun is surely a valid answer)
Every file chooser on your system suddenly knows about tags and gives you a way to query them.
Again not speaking to the project itself, but in principle achieving good performance on something like this would boil down to “do what SQL databases do, then design efficient schemas and queries for the use case”.
Fortunately you can try Supertag out without touching your original files, as Supertag will use symlinks to your original files, so you don't need to move them.
https://github.com/discoproject/discodb
Cool work!
It’ll work well for:
- Aggregate workflows (i.e. composing heterogeneous collections)
- Library curation (i.e. individual or mass categorization)
- Content discovery/rediscovery (i.e. find things based on metadata rather than location)
It’ll either poorly serve or need additional consideration for:
- Contained dependencies (i.e. any kind of encapsulated package that has internal references)
- Structural authorization (i.e. a given user may or may not be permitted access to some or all functions based on the branch/leaf they’re trying to access)
- Structural layout (i.e. the user has a hierarchy in mind which they wish to reify)