Sure, you can obviously see the parallels between a TPU and a GPU. I guess the main differentiator is that a TPU cannot and has not ever been able to do graphics, while a GPU is still pretending to be that. But the advantage of a TPU to Google is twofold: they don't have to pad Nvidia's profit margins (note, however, that they do have to pay partners like Broadcom to make them), and the hardware is co-evolving to be optimal for Google's specific workloads, while Nvidia has to go to market with something that tries to solve everyone's generic problems. Just as an example, when I was at Google the group I was in was working on integrating the then-new TPUv2 with the internal fleet-wide profiling infrastructure. The hardware people had added an absurd number of performance counters, so many that it was going to take an entire Skylake Xeon CPU to handle the flood of performance data. But that was considered worth it because that information was going to inform both the next generations of models, tools, and compilers, and the next generation hardware.