I think a big problem with metadata-aware “file systems” is that the metadata is lost once the file is exported out of the system. This is a problem with ID3 tags for instance.

Another problem is where you make the compromise in the no-mans land between fully fledged data structure and file system. As soon as you start adding meaningful metadata to the file system, it quickly becomes apparent that you want the files themselves to be structured data and not just opaque sequences of bytes. At that point you’re redesigning the OS since that mode of usage requires user and application buy-in. It’s just a tough design problem to make any universally applicable progress in this space and it seems like any sort of non-HFS system is destined for application specific use cases.

> the metadata is lost once the file is exported out of the system. This is a problem with ID3 tags for instance.

Are you sure? ID3 tags are embedded in the file itself, and therefore remains in the file no matter what medium you store it on.

That’s my point.

"I think a big problem with metadata-aware “file systems” is that the metadata is lost once the file is exported out of the system."

This is my thought as well. I used to run BeOS as my main OS, and when I finally had to move away from it, all the BeFS metadata was left behind as well.

> the metadata is lost once the file is exported out of the system

As someone who has implemented such a "file system" (or three) for various types of enterprise clients, some of which in turn serve it to other b2b clients of their own... I just have to say... this isn't necessarily a bad thing, and also, it's not necessarily true either. For starters, one can easily give every file a unique uuid, and map that uuid to a spreadsheet full of metadata. Additionally, a little vendor lock in to keep "special features" like management of a bespoke file system isn't necessarily a bad thing, either, if it's in your best interest to keep paying customers. Application specific use cases? Sure... but what isn't? You can build a generic abstract non-hierarchical file system though... easily.

I would recommend looking at some of the ideas that ReiserFS was trying to do -- and some of the ideas about metadata and expanding some of the concepts of filesystems were present in their ideas as well. In particular one idea they had was allowing you to do things like SQL searches in your filesystem, using filesystem plugins (effectively the idea is to allow for database structures to be stored using filesystem plugins so that searches and operations become just VFS operations).

Obviously we know what happened to Hans Reiser, but I've always felt that some of the clever ideas in reiser4 were not fully explored because of what happened.

Hello, I am trying to make an application for my wife to manage embroideries. I encounter almost all your issues. My wife has thousands of embroideries downloaded from internet. There are many duplicates (filenames not unique because of internationalisation and special characters). She needs to add tags to help search. She also needs groups of tags (tiger belongs to animals, ...). She also has metadata (origin of the file, which license applies to which file). Sometimes she modifies an embroidery. She needs to ensure that the original file is not modified and to keep a link between the two files. Sometimes there are groups of embroideries (for example letters) or there is documentation attached to an embroidery. It is a mess and I think that your work would help a lot to handle this kind of use cases. The current paradigm of directory tree is outdated and something smarter could be done.

How large are these embroidery files? I feel like this is something that a SQL database might be able to help with. It's probably not an ideal solution, but remember that a filesystem is really nothing more than a database, which organizes files on-disk in a particular way and indexes them so that you can find them. It obviously has serious limitations due to its hierarchical nature, which is why relational databases were invented, so using existing tools it's probably quite feasible to create an application that uses a SQL database to index all the embroidery files and store all this data on them (license, whether it's a derivative of another one, and tags), and then if they're too large to just store in the DB itself, just point to files in the regular filesystem.

To be pragmatic, it sounds like she needs a relational database more than a filesystem. (Whether or not filesystems should be more like relational databases is a hypothetical at present.)

If she actually manages that sort of data outside of an RDBMS I do suffer with her.

Your description is very similar to a problem I have with academic papers as PDFs. I was toying with `rsync` solution and naming conventions for original file names (as they were named when acquired) and renaming after review (EG. using the file system). Organizing the files with tags, collections would be a great improvement.

If she is using Windows 10 (or similar), I've found http://tabbles.net/ is a pretty cool solution.