It's rare we ever need the whole dataset - in fact, it's really great to be able to say to the user "don't send us your 100GB dataset: just go to the "acquisition" subfolder and send me the 10MB file called "oscilloscope.dat"". With HDF5 this is difficult enough that it's almost always easier to send 99.9% of useless data (i.e. the whole file) when all you want is a single array within it.
If your users will rarely need to do this, you could just store the entire folder hierarchy in a .zip and access it using standard tools that most programming languages have. It's worth noting that the new Microsoft Office formats do exactly this - in their case, a bunch of XML files inside a .ZIP. (Rename a .docx to .zip and you'll see!).
MATLAB has moved from their own custom binary format to HDF5, which is the lesser of two evils.
Yes, I do know about docx and zip.
I'm happy with what matlab has done. People send me .mat files and I happily process them in python. And my plots usually look much nicer also. :)
HDFfView is not the only viewer in town. There are several viewers.
I've used HDF5 weekly for the last 5 years and so have my associates and its been wonderful. MATLAB even uses it to store its .mat files these days.
I think you are misinformed and Im sticking to my story !