It is xor'd because all entropy sources are xor'd. That's just the design of /dev/random.
And because of this, Intel or whoever would need to have weakened each and every entropy source you use to make anything of it.
That's not true, because the RDRAND entropy is mixed in last. So once you're under the assumption that RDRAND is nefarious, the microcode only needs to detect the rdrand-to-xor pattern to make the entire entropy pool predictable (for example: by setting the non-rdrand input to the xor operation to zero it could disable all other entropy sources).
From a quick skim of current sources, much of that has recently been rolled back"
At the very least, that looks like there might be something in it. I don't know crypto well enough to say for sure, but it looks like Linus needs to re-read rand.h from what that article says about the state of the comments and previous rolled back changes from Intel. It does make one wonder if Linus' arrogance and rudeness could be used as cover for a genuine backdoor.