Edit: thinking about this on the way back from lunch, read-heavy workloads give you the best case for persistent data structures, but that's still the same as for mutable structures, i.e. no object churn. I can't think of any cases where persistent structures would create less garbage.
Persistent data structures allow the VM or compiler to determine when things need to be copied. Mutable structures require programmers to worry about copies, so they will make copies when they feel they might be necessary. In very well-defined cases a smart programmer can optimize this to a point where it probably uses less memory than a persistent data structure, but in the average case (and with the average programmer), the VM will likely do a better job of knowing when to copy and when to not bother.
Persistent data structures intrinsically involve copying on updates, and no amount of compiler cleverness can avoid that. I can see the possibility for compiler magic with stream fusion, for example, but I'm not seeing it for data structures. The only win I can see is if you have multiple clients of your data structure holding references to multiple versions, but even there the compiler doesn't help you. Am I missing something?
The compiler magic can help you there. You are looking for linear types.