It is possible to program in java without allocating GC at all. (if you do not count class loader and some other system stuff). Use Unsafe and primitive (on stack) variables. Peter has some nice demos on his blog.

but the on-heap references you keep are only going to be much smaller and thus, alleviate a lot of gc. Offheap storage tend to be things like caching, and caching in java does incur gc costs that you avoid with off-heap stores. It's not a bad solution.