Google's First Tensor Processing Unit: Architecture

Re:Google's First Tensor Processing Unit: Architecture by nl on Tuesday March 26, 2024 @02:01AM (#39824647)

Actually their real advantage is the large set of highly optimised CUDA kernels.

This is the thing that lets them outperform AMD chips even on inferior hardware. And the fact that anything new gets written for CUDA first.

There is OpenAI's Triton language for this too and people are beginning to use it (shout out to Unsloth here!).

> Reserved fab space.

While this is true, it's worth noting that the inference only Groq chip which gets 2x-5x better LLM inference performance is on a 12nm process.

[ Parent | on HN ]

Re:Google's First Tensor Processing Unit: Architecture by ants_everywhere on Tuesday March 26, 2024 @08:03AM (#39826801)

Honest question: will AI help AMD catch up with optimized CUDA/ROCM kernels of their own?

[ Parent | on HN ]