While it's true that you lose atomicity with this, it can both burn you and help you: you can track specific parts of your dataset in revision control, email subsets of it back and forth, combine datasets easily, or even store it across several servers and manage it with symlinks, for example.
Our users are aware of this and it isn't really a problem for our use-case. But if you're worried, there's always the option of having your whole dataset as a ZIP/TAR file (like all of Microsoft Office file formats are - XML files within a .ZIP); tools for modifying folders within ZIP are much more well-established than HDFView, and most modern programming languages provide libraries to read and modify files within archives without unzipping them; you could make your program agnostic to the files being within an archive (high portability, lower performance) or directly within the FS (loss of atomicity, easier/faster to use).
Even better directly use BTRFS (or ZFS) send / recv to send to move to another system with that file system.
Can also pipe tar directly to something like nc. Remember benchmarked moving a large number of files on a LAN and tar + nc came out as the winner over say rsync or scp. It saturated 1Gbps connection pretty close to its maximum expected capacity.
You'd obviously not expect user to do that by hand and would write tools to do it.
Another thing that will happen is this: if you just use a directory full of files as a single logical file, you will end up writing code that does the equivalent of 'rm -rf ${somedir}' because when users choose to overwrite "files" (really, a directory), you need to clear out any previous data so experiment runs don't get mixed. You can easily see where this can go bad; you will have to take extraordinary care.