In Java, every object stored on heap (and they can't be stored on stack) has a JVM-induced space overhead. Also, the allocator is free to spread the objects across the heap in a chaotic manner, which will make for less than optimal utilization of cache lines.

That's... not how JVMs work.

All modern VMs (not just limited to Java here!) apply two key optimizations. The first is escape analysis, which checks if references to objects will escape the current function boundary. If not, the objects will be stored on the stack instead of the heap. The second is generational GC, where memory allocation looks like this:

   void *new_ptr = heap_mem;
   heap_mem += alloc_size;
   if (heap_mem >= max_size)
     outlined_function_to_get_larger_blocks_of_memory();

It's actually likely to be a tighter allocator than C/C++'s malloc, since there's no mucking about with freelists.