Complete speculation, but does tmux use the xterm alternative screen buffer? I can see a small bug in that causing huge memory leaks, but not showing up in testing.