IME, it's not so much the amount of memory¹, as the pressure it places on the GC to keep up w/ these (often small, but numerous) allocations. Other languages can allocate and deallocate these types of structures with just a stack pointer move in both cases, since nothing ever leaves the stack. (I think Java's allocator is typically approximately or exactly as cheap as a pointer-move on allocation, but deallocation is another story.)

¹but I don't want to downplay this concern entirely; in the parent's case of an int needing boxing, that effectively doubles the cost: the int, and now the pointer to that int. (Plus any hidden requirements of the allocator, but even if we assume that's free, the story might still be plenty grim.)