> If your program has threads, unmapping memory is generally to be avoided.

Could you expand on this? I recently encountered a situation where allocating and freeing many large allocations using mmap() seemed to eventually cause problems with thread creation, but I had assumed that it was probably because the virtual address space had become too fragmented, which of course would not solely be a result of unmapping. Or maybe that fragmentation is what you're referring to, and I'm just reading too much into that sentence.

It just creates long stalls as the TLBs of all cores that threads in the process run on are flushed and must be reloaded from backing RAM.

mmap is an operating system interface, or at the very least a part of POSIX. it isn't part of the C++ standard, and so you cannot rely on it being present on systems that don't claim extensive POSIX compatibility. As for MAP_POPULATE and MAP_HUGETLB, try using that on a POSIX-ish OS that isn't Linux and see how far you get.

What, are there any left?

Every OS nowadays has mmap, by some spelling, and they all support hugepages, likewise. A program that doesn't use its host OS doesn't, generally, do much.

Perhaps you've not come across the idea of cross-platform development?

I have a large fairly successful cross-platform program that does a hell of a lot. All of our interactions with the OS are wrapped, either by libstdc++ or by the portability library that we use.

We would never allow a direct call to mmap() from the main application code ... we don't even allow direct calls to most of the POSIX API since we have to run on Windows too (without WSL or whatever MS' current POSIX layer is).

"By some spelling" is precisely why almost all programmers are better off using "new X[N]" than mmap, unless you actually think that code like this is good:

#ifdef __APPLE__ #define LOCAL_HUGETLB 0 #else #define LOCAL_HUGETLB MMAP_HUGETLB #endif ...

mmap (..., foo|bar|LOCAL_HUGETLB)

This cannot be a serious suggestion regarding the recommended way to allocate memory.

Seems pretty easy and reasonable to write a wrapper function for mmap for all 3 OSes in common use...

Exactly. There's nothing new about portability shims. Really nothing, been using them since the '80s.

> Every OS nowadays has mmap, by some spelling

I mean, I'm fairly sure that's the problem. (Plus a lot of systems support C++ but not virtual memory…)

Just to take one (small) example, the Arduino IDE is C++, and you will not mmap() a lot there.

You also won't new[] a lot there. How much can it even physically address?

I've actually used an STL port to Arduino and was able to create a standard vector of length three but not four. that crashes the system.

I believe it uses a custom allocator.

Actually Arduino code does tend to use dynamic allocation for things like strings - it's a lot easier than the alternative, they tend to have quite a lot of memory these days (e.g. 1 MB) and hobby projects don't need to be super-reliable usually.

One megabyte or ten isn't enough to bother with mapping, and on Arduino you don't typically have memory mapping hardware anyway. "Big" means gigabytes these days, sometimes hundreds of them.

but often you might write code that you dont know it will end up there, like lib

Many c stdlib will fallback to mmap above a certain threshold :)