> Hmm... if the allocator affects performance so much, then maybe the code simply uses the allocator too frequently?

I'm not familiar with musl's allocator so it's hard for me to say anything about it. However what is pretty common is that modern allocators use thread local allocators to avoid hammering global locks too much. Since Rust is very concurrent you don't need too many allocations to benefit from these optimizations.

Not sure its right to blame the code when changing the base image/allocator results in 30x performance improvements on the same hardware. I would be inclined to blame a poor choice of base image/allocator.

It's not a docker thing, we're getting the same 30x performance hit without using docker, simply compiling musl vs glibc binaries with the default allocator chosen by Rust.

Every optimization that you don't need to do is a win. With glibc, performance impact of unnecessary allocations might be minimal, but with musl allocations suddenly matter a lot.

In a lot of ways, rust definitely encourages heavy use of allocator.

Even if you program better for it, one of the prevailing opinions on “learning about lifetimes” is to “not learn lifetimes, just make copies”.

If the allocator can't keep up with the code, then maybe the allocator simply is bad?

> putting each C++ object into its own heap allocation behind a smart pointer, creating and destroying tiny objects all the time

Are C++ allocators so bad that they can't handle this use pattern? Yes, you can make a bunch of manual effort to use the allocator less, but you shouldn't have to.