News for hackers. Stuff that matters.
Re:Mimalloc – A compact general-purpose allocator by nh2 on Saturday June 22, 2019 @04:54PM (#20251675)
Are there functions available with which I can at run-time query how much OS memory is used, how much handed out in allocations, how many mmap()ed pools are used, and so on?

I find that one of the most important features of a malloc library to debug memory usage.

glibc has these functions (like malloc_info()) -- they are very bugged in that they return wrong results, but after patching them to be correct, they are super useful.

Re:Mimalloc – A compact general-purpose allocator by jasonzemos on Saturday June 22, 2019 @06:39PM (#20252080)
Any more info/link on how they are incorrect and the patches you need to fix them?
Re:Mimalloc – A compact general-purpose allocator by nh2 on Sunday June 23, 2019 @07:53PM (#20259491)
Sure, my patches are linked in the bugs I filed:

https://sourceware.org/bugzilla/show_bug.cgi?id=24026 - "malloc_info() returns wrong numbers"

https://sourceware.org/bugzilla/show_bug.cgi?id=21556 - "malloc_stats printing size_t fields as unsigned int"

If you're interested in this topic:

After finding one bug after the other due to 32-bit integer overflow in malloc.c, I just searched for "unsigned int" in that file for fun, and 30 seconds later found what I consider a security vulnerability in realloc():

https://sourceware.org/bugzilla/show_bug.cgi?id=24027

In certain situations, if you realloc() e.g. 32G + 5 bytes (reallocs this large can happen in large programs e.g. for data analysis), it'll copy only 5 bytes, and leave the rest as memory garbage.

That experience taught me that open source code being old doesn't mean anybody ever read it.

Re:Mimalloc – A compact general-purpose allocator by rurban on Monday June 24, 2019 @07:44AM (#20262485)
pt2malloc is only maintained by glibc, but not really. Since they cannot maintain it, they want to get rid of it. The upstream maintainer released a better version pt3malloc, which glibc refused to adopt. It needs one more word per alloc.
Re:Mimalloc – A compact general-purpose allocator by ndesaulniers on Sunday June 23, 2019 @12:37AM (#20253333)
On Linux, there's a bit of the info you're looking for under /proc/self/*. (Not a great answer; the info under /proc/ isn't exactly well structured, but doesn't change super often).
Re:Mimalloc – A compact general-purpose allocator by nh2 on Sunday June 23, 2019 @07:55PM (#20259497)
/proc doesn't really tell you much about userspace concerns such as "how much memory has the malloc implementation handed out to the program.

It describes only the kernel-userspace parts of memory usage. For example, malloc fragmentation is not inspectable in it.

(If you enable the mallopt() setting that turns each malloc() into a separate mmap(), then it tells you a lot, but that is not a default setting.)