The RNG attack suggested can pass verifiability while at the exact same time introduce a backdoor and thus constantly be on.
Faking consistency is likely impossible without causing a huge amount other calculations (which the CPU will do as part of day-to-day operations) to fail.
That did happen at one point, and cost Intel an estimated $475 Million[1]. Of course, that was (presumably) an unintentional bug.
[1]http://en.wikipedia.org/wiki/Pentium_FDIV_bug
--
I don't consider SMM or microcode updates to be above suspicion; Ken Thompson's's Reflections on Trusting Trust suggests that nothing is above suspicion.
Such a back door could be accomplished with tools like Dual_EC_DRBG (http://en.wikipedia.org/wiki/Dual_EC_DRBG).
Either they are just dumb and designed Dual_EC_DRBG by accident, or they design systems like that for a purpose. Your assessment of what they need is an assertion, based on what exactly? Security is designed not around what one thinks an attacker needs, or why they would want do do something. It's about preventing an attacker from getting access to things a person wishes to keep secure regardless of why they would want to. In that light weaknesses in random number generators is a relevant field of inquiry. Otherwise simply ignore cryptography altogether, trow up your ands and say why bother.
Everyone seems to be missing the point: RdRand is the least of your worries. If you can't trust the CPU, you can't trust it to multiply correctly, much less perform crypto.
If a CPU maker or OEM wanted to be evil, they wouldn't even need to bother backdooring the hardware. SMM or microcode updates would be much easier to compromise.
[1] https://mailman.stanford.edu/pipermail/liberationtech/2013-J...
[2] https://mailman.stanford.edu/pipermail/liberationtech/2013-J...