Re:Moving away from HDF5
by rossant on Saturday January 09, 2016 @06:45AM
(#10870921)
Unfortunately our users can't afford fancy computers with hundreds of GB of RAM. They often need to process entire datasets on laptops with 16GB of RAM but 1TB+ GB of disk space. Of course with 200 GB of RAM with have no problem at all...
Also, as I said, the 100,000 x 1000 example is a quite optimistic one, we do have cases now with 100,000,000 x 10,000 arrays, and this is only going to increase in the months to come with the new generation of devices.
Also, as I said, the 100,000 x 1000 example is a quite optimistic one, we do have cases now with 100,000,000 x 10,000 arrays, and this is only going to increase in the months to come with the new generation of devices.