That “simple bump allocator” is basically allocating everything on the thread's stack, as the GP mentioned.

It's what you put on the heap that has complex lifetimes. Sometimes you can fix that with an arena. If you can't, you probably can't figure out when the last reference to it dies either.