Re:Mio – Cross-platform header-only C++11 library for memory...
by gmueckl on Saturday October 20, 2018 @12:20PM
(#18263962)
I think that this is the wrong way to use mmap. Just map the whole file at once. The operating system will automatically read the pages you access from disk. And if memory gets tight, these pages will be flushed to disk if they are dirty and then discarded before the system starts paging. These mmapped pages essentially live in the disk cache.
[ Parent
| on HN ]
Re:Mio – Cross-platform header-only C++11 library for memory...
by mandreyel on Saturday October 20, 2018 @12:53PM
(#18264124)
Hmm, I see, I haven't come across this suggestion but I may not have researched this topic well enough. I'll try your suggestion next time.
I appreciate the responses--learned something new today!
Re:Mio – Cross-platform header-only C++11 library for memory...
by quotemstr on Saturday October 20, 2018 @12:32PM
(#18264020)
> The operating system will automatically read the pages you access from disk. And if memory gets tight, these pages will be flushed to disk if they are dirty and then discarded before the system starts paging.
You can tell that you understand how modern OS memory management works when you realize that the OS "automatically read[ing] the pages...from disk" and "flush[ing them] to disk" on memory pressure is paging whether those pages are anonymous pages or mmaped file pages. :-)
[Edit: flushing dirty file-paged pages is analogous to swapping anonymous memory to the swapfile. Discarding clean file-backed pages is a bit like discarding anonymous pages that have been made unused through munmap, process death, etc.]
But to the GP's point: you don't need (except to conserve address space) to limit file mapping size. I think he really wants something like MADV_FREE. But it's complicated.
Re:Mio – Cross-platform header-only C++11 library for memory...
by gmueckl on Saturday October 20, 2018 @12:47PM
(#18264095)
Therr is a subtle difference between anonymous and mapped readonly pages: the later can be discarded right away because their contents were read from permanemt storage to begin with. Anonymous pages need to be written to disk first and that is significantly slower.