C# suffers from that too. I've personally seen a prod instance experience OOM with something like 40% free virtual memory. There's a reason why ArrayPool is part of mscorlib.

All of that said, x64 greatly reduces the effects of memory fragmentation: there's always more virtual memory.