A GPU can switch threads on a cache miss, something few CPUs can do. Even the CPUs that do this switch between 2-8 threads (it's called SMT architecture, for example, popular Intel Pentium/Core CPUs have at most 2 hardware threads per core, Xenon Phi has 4). GPUs have dozens of threads per core and, often, more cores than CPUs. Combined with the fact that a GPU thread is much wider than a CPU this allows a GPU to saturate memory bandwidth even on a pointer chasing code. The same code causes the CPU to stall and underutilize its memory bus (as soon as the few threads hit cache miss there is nothing to do, software multithreading cannot switch between threads that are waiting for data).
Of course, as I said, it's still not efficient even on GPU (most of the bandwidth is used to move unneeded data as you only need one word from the whole cache line) and you still need to write massively parallel code.
Yes, but if you actually try to do it, you'll find that "massively parallel" code can be written only for certain problems that lend themselves to being, well, "massively parallel".
I've seen people struggling with GPGPUs ever since they appeared (at a certain point in my career I was involved in heavy number-crunching research). Six years later, there are still a lot of real-life problems that can't be put in a GPGPU-useful framework; in fact, coming up with good models so that you can reason if a given problem would ever lend itself to being solve like that is one of the more important progresses of the scientific community.
Look, GPGPUs are cool and all, and useful for certain classes of problems, but let's not do the whole Itanium thing all over again, please.
I said that APUs are the future of computing not because I'm favoring parallel applications, but because boring old manufacturing constraints are making greater single-threaded performance more difficult to achieve. What we have now is about as good as it's going to get, at least until we see 3D chips or molecular nanotechnology.
A whole lot of real-life stuff is either parallel or does not need heavy computing. This is why people started building massively parallel computers well before the last 6 years[0].
Exactly. A lot of the GPGPU pitch is easily refuted with some hindsight.