News for hackers. Stuff that matters.
Re:Memory is slow, Disk is fast – Part 2 by mastax on Thursday September 04, 2025 @09:48PM (#45134269)
MAP_HUGETLB can't be used for mmaping files on disk, it can only be used with MAP_ANONYMOUS, with a memfd, or with a file on a hugetlbfs pseudo-filesystem (which is also in memory).
Re:Memory is slow, Disk is fast – Part 2 by inetknght on Thursday September 04, 2025 @10:28PM (#45134451)
> MAP_HUGETLB can't be used for mmaping files on disk

False. I've successfully used it to memory-map networked files.

Re:Memory is slow, Disk is fast – Part 2 by minitech on Thursday September 04, 2025 @11:07PM (#45134638)
That doesn’t sound like the intended meaning of “on disk”.
Re:Memory is slow, Disk is fast – Part 2 by inetknght on Thursday September 04, 2025 @11:11PM (#45134653)
Kernel doesn't really care about "on disk", it cares about "on filesystem".

The "on disk" distinction is a simplification.

Re:Memory is slow, Disk is fast – Part 2 by pclmulqdq on Thursday September 04, 2025 @11:54PM (#45134845)
The kernel absolutely does care about the "on disk" distinction because it determines what driver to use.
Re:Memory is slow, Disk is fast – Part 2 by ddtaylor on Friday September 05, 2025 @01:23AM (#45135217)
The interface is handled by the kernel.
Re:Memory is slow, Disk is fast – Part 2 by loloquwowndueo on Thursday September 04, 2025 @10:59PM (#45134599)
Share your code?
Re:Memory is slow, Disk is fast – Part 2 by inetknght on Thursday September 04, 2025 @11:07PM (#45134637)
I don't work there any more (it was a decade ago) and I'm pretty busy right now with a new job coming up (offered today).

Do you have kernel documentation that says that hugetlb doesn't work for files? I don't see that stated anywhere.

Re:Memory is slow, Disk is fast – Part 2 by Sesse__ on Friday September 05, 2025 @05:12AM (#45136551)
It's filesystem-dependent. In particular, tmpfs will work. To the best of my knowledge, no “normal” filesystems (e.g., ext4, xfs) will.
Re:Memory is slow, Disk is fast – Part 2 by inetknght on Friday September 05, 2025 @09:57PM (#45145898)
It works fine on my ext4 fs...
Re:Memory is slow, Disk is fast – Part 2 by inetknght on Friday September 05, 2025 @01:06PM (#45140890)
My bad, don't use `MAP_HUGETLB`, just use `MAP_HUGE_1GB`.

See a quick example I whipped up here: https://github.com/inetknght/mmap-hugetlb

Re:Memory is slow, Disk is fast – Part 2 by squirrellous on Friday September 05, 2025 @02:38AM (#45135603)
This is quite interesting since I, too, was under the impression that mmap cannot be used on disk-backed files with huge pages. I tried and failed to find any official kernel documentation around this, but I clearly remember trying to do this at work (on a regular ECS machine with Ubuntu) and getting errors.

Based on this SO discussion [1], it is possibly a limitation with popular filesystems like ext4?

If anyone knows more about this, I'd love to know what exactly are the requirements for using hugepages this way.

[1] https://stackoverflow.com/questions/44060678/huge-pages-for-...

Re:Memory is slow, Disk is fast – Part 2 by bawolff on Friday September 05, 2025 @06:06AM (#45136878)
Trying to google this i found https://lwn.net/Articles/718102/ which suggests that there was discussion about it back in 2017. But i can't find anything else about it except a patchset that i guess wasnt merged (?). So maybe it was just a proposal that never made it in.

Honestly i never knew any of this i thought huge pages just worked for all of mmap.

Re:Memory is slow, Disk is fast – Part 2 by inetknght on Friday September 05, 2025 @01:05PM (#45140880)
My bad, don't use `MAP_HUGETLB`, just use `MAP_HUGE_1GB`.

See a quick example I whipped up here: https://github.com/inetknght/mmap-hugetlb

Re:Memory is slow, Disk is fast – Part 2 by squirrellous on Saturday September 06, 2025 @03:58AM (#45147460)
Cool! Thanks for the example. The aforementioned work thing requires MAP_SHARED as well which IIRC is the reason it would fail when used together with files and huge pages, but private mappings work as you show.
Re:Memory is slow, Disk is fast – Part 2 by inetknght on Friday September 05, 2025 @01:05PM (#45140875)
My bad, don't use `MAP_HUGETLB`, just use `MAP_HUGE_1GB`.

See a quick example I whipped up here: https://github.com/inetknght/mmap-hugetlb

Re:Memory is slow, Disk is fast – Part 2 by bawolff on Friday September 05, 2025 @04:01PM (#45142964)
The mmap man page kind of implies that would be a no-op, but i haven't tested myself.
Re:Memory is slow, Disk is fast – Part 2 by jared_hulbert on Friday September 05, 2025 @02:05PM (#45141621)
Adding MAP_HUGE_1GB and not MAP_HUGETLB does compile and run for me. Not convinced that its' actually doing anything. Performance is the same.
Re:Memory is slow, Disk is fast – Part 2 by inetknght on Friday September 05, 2025 @02:18PM (#45141792)
Well now that it works, feel free to start poking around at it for a follow-up blog post :)
Re:Memory is slow, Disk is fast – Part 2 by mananaysiempre on Friday September 05, 2025 @02:38AM (#45135606)
It looks like there is in theory support for that[1]? But the patches for ext4[2] did not go through.

[1] https://lwn.net/Articles/686690/

[2] https://lwn.net/Articles/718102/