For funzies, I looked at one our ActiveMQ Message brokers, which are ran on 1g machines with a 640m heap. The heaps are fully allocated and the resident set size is ~658m. Linux reports it has 66M of free memory and 95M of available ram. 440m of swap is being used and is only lightly being paged out under light load of 400 msgs/s. This same setup can push 2000msg/s before GC thrashing becomes an issue and we need more heap.

So no, I don't think there's any truth to the claim "Hotspot always uses an extra 256m of memory". Sounds like the kernel is doing what it's supposed to: free memory is wasted memory.