All good so far, for (say) HPC where you have people with guns guarding the machines you can turn on all the performance. The issue is that as CPUs have got more complicated, it's increasingly easy to attack the internal state of the processor. This is already possible in a single thread (i.e. the prototypical spectre and meltdown implementations), however imagine what you can do next door to a completely different "core". The processor maintains separation of structure but not of state, so you can use (say) timing side channels to extract information about what your neighbour is up to.

FWIW these issues are more of a problem for Cloud vendors than you or I as they usually require pretty specific knowledge of the hardware being attacked and even then are not easy to pull off.