Context-switching is expensive. You have to swap out the data being worked on by thread #1 and swap in the data for thread #2. So you end up being bottlenecked by memory bandwidth and latency rather than by raw compute.