Compare that to something like VNC which IIRC simply sends the screen data over as a raw bitmap and must rely on (potentially lossy) compression in order to get reasonable speed. This of course has the advantage that the client software needs to be much less intelligent to understand it and means that it needs much less deep OS integration. This is why VNC clients and servers are available and can work easily between pretty much any platform.
The GP claims that X11 is capable of doing the same optimisations that RDP can do but this is not utilised well because the windowing toolkits such as GTK render their output to some other area of memory and X only gets to see the bitmapped result.
Not sure how teamviewer works, but I know some of these paid services use intermediaries to trasmit data over faster internet links to reduce latency but of course this costs them money so they have to charge you.
SO discussion here: http://stackoverflow.com/questions/9498877/how-is-teamviewer...