Right, I should have mentioned GPUs have a ton of hardware threads. Then again, they have to, GDDR5 memory access can take a microsecond. Try latency like that on a generic CPU, hyperthreading or not...
So: GPUs are wide SIMD machines with a lot of hardware threads, massive branch and glacial memory latencies. When there's a branch or memory latency, HW simply switch thread. GPUs don't care about serial execution performance.
That's not really true; you can kind of treat it that way, depending on the algorithm (eg, matrix multiplication breaks down cleanly that way), but there are serious flexibility advantages in the "single-instruction-multiple-thread" model vs SIMD. For example, consider streaming large numbers of hash lookups - difficult to express clearly with a pure vector processing model.