GPUs can just do fast matrix multiplications in parallel, but it’s up to the developer to reframe their software into matrix multiplications. For some applications it’s great, but for most it isn’t realistic.

Also if General processors are now hitting a wall, it’s only a matter of time before GPUs do.