Parallelization and GPUs were the hot story 10-5 years ago, and require(d) a pretty substantial shift in the software stack for less-general gains. You're still hoping the cost-per-transistor goes down. I think recent 400W+ GPUs have shown that we're coming close to the end of this particular S-curve. The big question is whether any of the tricks we have left are broad enough and strong enough to address the economic problem.