In the new tcmalloc (and, I think, hoard?) the fastest pools are essentially slabs with bump allocation, so the fastest (and by far, the most common) calls are a grand total of 15 or so instructions, without many cache misses (size class lookups tend to stay in the cache). Call overhead can be a substantial chunk of that.
But that’s not the whole story. Malloc perf is about what happens on free and what happens when the program accesses that memory that malloc gave it.
When you factor that all in, it doesn’t matter how many instructions the malloc has. It matters whether those instructions form a bad dependency chain, if they miss cache, whether the memory we return is in the “best” place, and how much work happens in free (using the same metrics - dependency chain length and misses, not total number of instructions or whether there’s a call).
This is only thinking at the hardware layer. There are also effects at the software layer.
Function calls not known to LTO prevent all sorts of compiler optimizations as well. In addition, as Jeff says on this thread, not being able to inline free means you get no benefit from sized-delete, which is a substantial improvement on some workloads. (I'd cite the exact number, but I'm not sure Google ever released it.)
Source: I literally worked on this.
(parent: read username :))
Agree, I shouldn't have been loose wrt the distinction between inlining and LTO. I think I also ran that experiment with inlining sized-deletes and came to the same (surprising, to me) conclusion.
I buy the sized delete win, I’ve seen that too, in some niche cases. But that’s a far cry from the claim that not inlining malloc is bananas. If that’s all you got then this sounds like a stunning retreat from the thesis of Jeff’s post. You’re just saying that you’ve got a free that benefits from size specialization, which has nothing to do with whether malloc got inlined.
int *a = ...;
int b = *a;
malloc(42);
b = *a;
printf(b);
Interestingly, the compiler emits no call to malloc, even. What!?
Without cheating:
It doesn't optimize the two stores to b away.
"Profiling a warehouse-scale computer" (by S. Kanev, et. al.) showed several % of CPU usage allocating and deallocating memory. While a malloc and free does have some data dependencies, the indirect jump (by dynamically linking) is an avoidable cost on the critical path by statically linking.
Perhaps Spectre mitigations have changed things, but we're still talking about a fraction of a fraction. As others have said, the best way to improve malloc performance is to not use malloc. Second best would be avoid allocations in critical sections so the allocator doesn't trash your CPU's prediction buffers.
You might argue that we should have done that optimization on our own, but we are just a small team doing an inhouse thing so we don't have those kind of resources, so to us it helps a lot that the the default malloc is very optimized.
I have thought the same for a long time until I ran some benchmarks on some of our 3D modeling code that uses Open CASCADE, and found out that in some cases allocation overhead was over 20% of total time spent in the CAD kernel...
Your data is legitimate, but you've got to ask yourself, would Jeff specifically and Google more generally care so much about it if it were also true for their use-cases?
This is Bayesian logic (our prior is that the systems engineers at Google are not incompetent), not an appeal to anything.
HN could indeed use more epistemic uncertainty and gray (vs black-and-white) thinking.