News for hackers. Stuff that matters.
Re:Loading NumPy arrays from disk: mmap() vs. Zarr/HDF5 by mborg on Thursday February 27, 2020 @09:14AM (#22433385)
The article leaves out a pretty important question you should ask yourself: "What fraction of the data will I need to access?"

If the answer is "much less than the whole file", accessing the data over the network makes sense. Otherwise, you should almost certainly bring it to a local disk, where mmap shines.

There are no comments on this comment yet.