Re:When your data doesn’t fit in memory: the basic techniques
by faizshah on Tuesday November 12, 2019 @03:46AM
(#21512589)
> it can sometimes have much worse performance than an explicit chunking strategy
Can you expand on this?
I haven’t heard this before.
Edit: Found a comprehensive discussion on this https://stackoverflow.com/questions/45972/mmap-vs-reading-bl...
mmaping is a form of uncontrolled chunking, where chunk size and location is determined by operating system's filesytem caching policy, readahead heuristics, and the like. As a result, it can sometimes have much worse performance than an explicit chunking strategy, especially if you just treat it as magic.
Or to put it another way, mmaping is helpful but you still need to understand why it might help and how to use it.