That's something you can do outside of prime95; the schedutil scheduler does that by default, which I believe is why I had hard crashes with ondemand and not schedutil.

(Ok, it actually ramps up frequencies slower rather than lock out cores, but the effect is the same.)