Yeah, in X anyone can write a keylogger. Or do something really useful with it :-)

(Personally I don't think this is a problem: if the attacker can run programs it's game-over already; just replace "su", "sudo", "firefox", or whatnot with your wrapper script which logs stuff, add ~/.local/bin to PATH by frobbing with shell rc, and presto)