I think I heard of a timing attack that was introduced by CPU optimizations, not in the underlying code at all! But I can't remember what research that was and maybe I'm confusing two different issues.

Certainly plausible. If the CPU optimizes one path but not another, in principle that's some information. In practice, gathering enough data to get that above the noise floor and turning it into something useful besides would certainly be difficult but maybe not too difficult.

I might be thinking of some of the cache timing issues about sharing physical hardware with other guest VMs. There were papers in the last two years showing circumstances in which another VM on the same device can learn about secrets in caches via timing experiments. But that's a cache issue rather than a general pipeline issue. So I still can't remember if there's something else that I'm thinking of or if I'm just confusing it with the cache stuff.

http://blog.cryptographyengineering.com/2012/10/attack-of-we...

More recent stuff in this vein:

https://eprint.iacr.org/2014/435.pdf

(Yikes!)

Well, the most obvious form of that is cache.

There's a entire class of timing attacks that rely on the CPU cache - they would not exist if CPUs didn't do the optimization of storing local copies of limited sections of RAM.