How fast can you allocate a large block of memory in C++?

Re:How fast can you allocate a large block of memory in C++? by PaulDavisThe1st on Thursday January 16, 2020 @12:42AM (#22061977)

mmap is an operating system interface, or at the very least a part of POSIX. it isn't part of the C++ standard, and so you cannot rely on it being present on systems that don't claim extensive POSIX compatibility. As for MAP_POPULATE and MAP_HUGETLB, try using that on a POSIX-ish OS that isn't Linux and see how far you get.

[ Parent | on HN ]

Re:How fast can you allocate a large block of memory in C++? by ncmncm on Thursday January 16, 2020 @12:57AM (#22062030)

What, are there any left?

Every OS nowadays has mmap, by some spelling, and they all support hugepages, likewise. A program that doesn't use its host OS doesn't, generally, do much.

[ Parent | on HN ]

Re:How fast can you allocate a large block of memory in C++? by PaulDavisThe1st on Thursday January 16, 2020 @01:03PM (#22067114)

Perhaps you've not come across the idea of cross-platform development?

I have a large fairly successful cross-platform program that does a hell of a lot. All of our interactions with the OS are wrapped, either by libstdc++ or by the portability library that we use.

We would never allow a direct call to mmap() from the main application code ... we don't even allow direct calls to most of the POSIX API since we have to run on Windows too (without WSL or whatever MS' current POSIX layer is).

"By some spelling" is precisely why almost all programmers are better off using "new X[N]" than mmap, unless you actually think that code like this is good:

#ifdef __APPLE__ #define LOCAL_HUGETLB 0 #else #define LOCAL_HUGETLB MMAP_HUGETLB #endif ...

mmap (..., foo|bar|LOCAL_HUGETLB)

This cannot be a serious suggestion regarding the recommended way to allocate memory.

[ Parent | on HN ]

Re:How fast can you allocate a large block of memory in C++? by smaddox on Thursday January 16, 2020 @08:19PM (#22070940)

Seems pretty easy and reasonable to write a wrapper function for mmap for all 3 OSes in common use...

[ Parent | on HN ]

Re:How fast can you allocate a large block of memory in C++? by ncmncm on Friday January 17, 2020 @04:56PM (#22079870)

Exactly. There's nothing new about portability shims. Really nothing, been using them since the '80s.

[ Parent | on HN ]

Re:How fast can you allocate a large block of memory in C++? by saagarjha on Thursday January 16, 2020 @01:47AM (#22062190)

> Every OS nowadays has mmap, by some spelling

I mean, I'm fairly sure that's the problem. (Plus a lot of systems support C++ but not virtual memory…)

[ Parent | on HN ]

Re:How fast can you allocate a large block of memory in C++? by IcePic on Thursday January 16, 2020 @02:47AM (#22062437)

Just to take one (small) example, the Arduino IDE is C++, and you will not mmap() a lot there.

[ Parent | on HN ]

Re:How fast can you allocate a large block of memory in C++? by ncmncm on Thursday January 16, 2020 @03:31AM (#22062649)

You also won't new[] a lot there. How much can it even physically address?

[ Parent | on HN ]

Re:How fast can you allocate a large block of memory in C++? by dekhn on Thursday January 16, 2020 @11:00AM (#22065717)

I've actually used an STL port to Arduino and was able to create a standard vector of length three but not four. that crashes the system.

I believe it uses a custom allocator.

[ Parent | on HN ]

Re:How fast can you allocate a large block of memory in C++? by IshKebab on Thursday January 16, 2020 @07:37AM (#22063913)

Actually Arduino code does tend to use dynamic allocation for things like strings - it's a lot easier than the alternative, they tend to have quite a lot of memory these days (e.g. 1 MB) and hobby projects don't need to be super-reliable usually.

[ Parent | on HN ]

Re:How fast can you allocate a large block of memory in C++? by ncmncm on Friday January 17, 2020 @05:00PM (#22079914)

One megabyte or ten isn't enough to bother with mapping, and on Arduino you don't typically have memory mapping hardware anyway. "Big" means gigabytes these days, sometimes hundreds of them.

[ Parent | on HN ]

Re:How fast can you allocate a large block of memory in C++? by alehander42 on Thursday January 16, 2020 @10:10AM (#22065186)

but often you might write code that you dont know it will end up there, like lib

[ Parent | on HN ]