> So the only advantage of the TPU is it's a simpler and more specialized asic

And everything that entails: lower energy consumption, higher throughput, lower cost at volume, higher profits for GCP, etc.

> Google didn't break any new ground in terms of training perf?

Relative to GPUs, sure, but I can't say how well they stack up against other custom ASICs for DL applications.