Caching effects. If you have an x-thread CPU and you split your workload into y >> x parts, it might happen that suddenly a workload fits into a faster CPU cache level (e.g. L1 instead of L2) and then you see huge speedups. I've seen things speed up manifold when I had 67 threads and added one more even though it was an 8-thread CPU. Keep that in mind when you do threading.