Not in the presence of DVFS, it turns out: https://www.hertzbleed.com/hertzbleed.pdf
AES cache-timing was broken over a network (but required, like, 2^28 samples).
I wouldn't bet the farm on this line of thinking providing resilience. It might be just annoying enough for an attacker to not really bother. (Or maybe only if the target is, like, a software cryptocurrency wallet with enough value to loot if they're successful.)
Recently, people have come up with partitioned caches to deal with this. I don't know if they exist in production. A simple strategy is turning off shared caches while running processes of different, security levels on their own cores. Also, investing in multi-core and many-core architectures for this.
Finally, many of us pushed for randomized execution or scheduling to throw off the timing of specific things. Combined with fine-grained processes (eg separation kernels), that should reduce what they can do a lot.