News for hackers. Stuff that matters.
Re:Drop millions of allocations by using a linked list (2015) by kqr on Friday March 12, 2021 @07:17AM (#26434954)
Doesn't even malloc do this under the hood by only occasionally calling sbrk and otherwise handing out blocks from an internally managed contiguous region?

What's the point of doing this again, manually? Or am I mistaken in my memory of the common modern malloc implementations?

Re:Drop millions of allocations by using a linked list (2015) by JoeAltmaier on Friday March 12, 2021 @08:46AM (#26435745)
It can be an order of magnitude faster to do it yourself, depending on how balled up the runtime library has gotten. If you can make some simple guarantees like "I'll realloc similar sized blocks mostly" and "I'm single-threaded thru these calls" then you can drop the bulk of the runtime drama.
Re:Drop millions of allocations by using a linked list (2015) by sly010 on Friday March 12, 2021 @10:00AM (#26436477)
What's the point of buying a jeep wrangler (or a ferrari), when you already have a subaru? Don't they all have 4 wheels?

General purpose tools are good for most cases. And being widely available they are cheaper too. Sometimes, however, you can do a better job with something special purpose tuned to the exact task at hand, but you have to buy the special purpose tool. The question is not IF it's better the question is whether it's worth the cost to you.

here are some things malloc can't do (afaik): - different pools for different parts (threads / datastructures / etc) of your app - better control over data locality - real time / non blocking allocations (i.e. low latency audio applications) - querying remaining available memory - releasing entire pools with a single call

Re:Drop millions of allocations by using a linked list (2015) by scott_s on Friday March 12, 2021 @12:12PM (#26438044)
Often, yes, you're better off just using the general purpose memory allocator.

I think the replies to your comment are unfairly giving you a hard time. Modern memory allocators are quite good. Before spending the time to roll your own allocation strategy, first do the experiments and analysis to convince yourself that memory allocation is even a performance problem. Once you've done that, do some work to implement a naive allocation strategy yourself and compare it against the standard allocator. If you're not outperforming it by a lot, work on it for a week. If after that week you're still not outperforming the standard allocator by a lot, just use the standard allocator.

Yes, sometimes optimizing for your exact use case can yield performance benefits. Buuuuuut... the chances that your use case actually falls out of what the general allocators have been designed for is small. And the people implementing the general purpose allocators are really good. They've optimized things you're not even aware of yet.

One random tidbit actually in favor of doing your own allocation: malloc and free are (in most compilers, I believe) always going to be function calls. Sometimes the benefit you get from rolling your own is just the benefit of eliminating a function call and inlining allocation code. ¯\_(ツ)_/¯

Re:Drop millions of allocations by using a linked list (2015) by cnity on Friday March 12, 2021 @07:23AM (#26435010)
malloc aligns very conservatively (because it doesn't know the size of the data you're allocating for), so there are some space gains to be had there.

AFAIK also malloc will call sbrk quite a lot, at least from my cursory reading of this musl implementation[0].

[0] https://git.musl-libc.org/cgit/musl/tree/src/malloc/malloc.c...

Re:Drop millions of allocations by using a linked list (2015) by CyberDildonics on Friday March 12, 2021 @09:06AM (#26435920)
That doesn't contain calls to sbrk at all, it maps memory in with mmap which is typical of every modern memory allocator.
Re:Drop millions of allocations by using a linked list (2015) by quantumofalpha on Friday March 12, 2021 @01:53PM (#26439404)
It uses both brk/sbrk ("__brk()" in that code) and mmap, the latter for allocating larger bits of memory. GNU libc does the same thing from what I remember from straces.
Re:Drop millions of allocations by using a linked list (2015) by cnity on Friday March 12, 2021 @09:51AM (#26436383)
You're right, I misinterpreted the calls to __brk