>If metadata doesn't fit in the fs view, then perhaps store them in hidden files next to the actual files

Sure, or in tags within the files like almost every music player, but that doesn't preclude there being a media library which is more like an interface to/cache of that metadata.