On the Hardware Wheel of Reincarnation [1] we are definitely swinging back into the realm of special-purpose processors and processor extensions all over the place. The gigahertz scaling free lunch has been dead for over a decade. We're still adding more transistors, but finding it harder to speed up day-to-day serial code. So, instead we're adding dedicated hardware for the common, hard stuff and throwing special-purpose chips into every nook and cranny of everything around the periphery of the main thread on the main CPU.

[1] https://www.computerhope.com/jargon/w/wor.htm

We are also inventing new ways to do computation. Massive deep neural nets require different hardware, and it seems there’s plenty of room to improve and experiment there.