why are heap allocations bad for performance?
1. You have to call to your allocator. Then do some type of search for free memory, of the approximate size. Then flag this memory as used. Then mark the remaining memory in that block as unused. Then mark the internal books to match the new layout.This is done efficiently, but modifying these collections take time. AND IT STILL FASTER THEN:
2. Calling the kernel to MMAP in new virtual memory, adding that to the pool, and well restarting this process all over again.
Allocation time is a big cost, and there is work to make allocation lazy by default in the Rust Compiler at the minute.
Also per thread pooling doesn't use more memory then not. In some cases it actually uses less. Citation: https://people.freebsd.org/~jasone/jemalloc/bsdcan2006/jemal...
Sure, the allocator itself can be expensive, and that's certainly important if you're doing "too many" allocations, but in general I think it's worse caching that matters the most.
To other programmers not so familiar with CPU caching, resharing two blurbs that helped me get a start of a grasp on it:
- Why do CPUs have multiple cache levels? https://news.ycombinator.com/item?id=12245458 , https://fgiesen.wordpress.com/2016/08/07/why-do-cpus-have-mu...
- CppCon 2014 - Mike Acton: Data-oriented design and C++ , https://www.youtube.com/watch?v=rX0ItVEVjHc
It's really a shame more platforms don't show cache/icache misses in an easy to access manner.
→ Systems programming newbie question: why are heap allocations bad for performance? Is it the additional level of indirection? The cost of calling your memory allocator? Something else?
My background, if that helps focusing answers: python/js programmer, did a tiny bit of C/C++, am ~approximately~ familiar with the stack (call frames, each with its context) vs. the heap (where to allocate memory for big/long-lived objects e.g. arrays and trees).