HDFfView is not the only viewer in town. There are several viewers.
I've used HDF5 weekly for the last 5 years and so have my associates and its been wonderful. MATLAB even uses it to store its .mat files these days.
I think you are misinformed and Im sticking to my story !
It's rare we ever need the whole dataset - in fact, it's really great to be able to say to the user "don't send us your 100GB dataset: just go to the "acquisition" subfolder and send me the 10MB file called "oscilloscope.dat"". With HDF5 this is difficult enough that it's almost always easier to send 99.9% of useless data (i.e. the whole file) when all you want is a single array within it.
If your users will rarely need to do this, you could just store the entire folder hierarchy in a .zip and access it using standard tools that most programming languages have. It's worth noting that the new Microsoft Office formats do exactly this - in their case, a bunch of XML files inside a .ZIP. (Rename a .docx to .zip and you'll see!).
MATLAB has moved from their own custom binary format to HDF5, which is the lesser of two evils.
Yes, I do know about docx and zip.
I'm happy with what matlab has done. People send me .mat files and I happily process them in python. And my plots usually look much nicer also. :)
Are you not concerned about i-node consumption with the file-system based approach?
I kind of the like the approach suggested above - if you have many small files store them in a zip archive and use some library to access the data directly.
"When in the field and the system shits the bed", to quote you... you can just open the dataset in Windows Explorer. Or Mac OS Finder. Or Nautilus. Or using 'cd' and 'ls' in Linux. Want to look at an array? Sure, open it in a hex editor or Python or, heck, FORTRAN88. You can .tar up your folder (or subfolder, or any arbitrary subset of the dataset) and send it to someone with no knowledge of the format whatsoever, and they'll be able to make sense of it in minutes. This isn't anything remotely complex - it's just using the filesystem rather than creating a filesystem-within-a-filesystem. Want to keep track of changes in your massive dataset? Sure, just back it up on Mac OS X Time Machine, or a simple rsync script, or even Git LFS.
Most researchers aren't computer scientists; they know how to use Dropbox and Notepad++ and open text files, and they don't want to have to install a Java-based HDF5View when they could just use Windows Explorer.
It's not even that HDF5 is that bad, it's just that filesystems are, in many respects, so much better.
(If it wasn't clear from the article, we're not just misinformed - we're making this call after having developed an entire software suite around HDF5, spent about two years of firefighting HDF5 issues and wasted days of development time (so many horror stories) - this is actual feedback from several dozen users, thousands of datasets, and petabytes of data.)