Perhaps this chapter can help? https://jax-ml.github.io/scaling-book/tpus/

It's a chip (and associated hardware) that can do linear algebra operations really fast. XLA and TPUs were co-designed, so as long as what you are doing is expressible in XLA's HLO language (https://openxla.org/xla/operation_semantics), the TPU can run it, and in many cases run it very efficiently. TPUs have different scaling properties than GPUs (think sparser but much larger communication), no graphics hardware inside them (no shader hardware, no raytracing hardware, etc), and a different control flow regime ("single-threaded" with very-wide SIMD primitives, as opposed to massively-multithreaded GPUs).

Thank you for the answer! You see, up until now I had never appreciated that a GPU does more than matmuls... And that first reference, what a find :-)

Edit: And btw, another question that I had had before was what's the difference between a tensor core and a GPU, and based on your answer, my speculative answer to that would be that the tensor core is the part inside the GPU that actually does the matmuls.

Sure, you can obviously see the parallels between a TPU and a GPU. I guess the main differentiator is that a TPU cannot and has not ever been able to do graphics, while a GPU is still pretending to be that. But the advantage of a TPU to Google is twofold: they don't have to pad Nvidia's profit margins (note, however, that they do have to pay partners like Broadcom to make them), and the hardware is co-evolving to be optimal for Google's specific workloads, while Nvidia has to go to market with something that tries to solve everyone's generic problems. Just as an example, when I was at Google the group I was in was working on integrating the then-new TPUv2 with the internal fleet-wide profiling infrastructure. The hardware people had added an absurd number of performance counters, so many that it was going to take an entire Skylake Xeon CPU to handle the flood of performance data. But that was considered worth it because that information was going to inform both the next generations of models, tools, and compilers, and the next generation hardware.

I mean yes. But GPU's also have a specific optimization, for graphics. This is a different optimization.

Did you also read just after that "without hardware for rasterisation/texture mapping"? Does that sound like a _G_PU?

You asked a question, people tried to help, and you lashed out at them in a way that makes you look quite bad.