News for hackers. Stuff that matters.

A comment on We fine-tuned Llama 405B on AMD GPUs. Parent comment

Re:We fine-tuned Llama 405B on AMD GPUs by jgalt212 on Monday September 23, 2024 @06:42PM (#41631362)
Is there some cost rule of thumb to compare Nvidia, AMD, and Google TPU?
[ Parent | on HN ]
Re:We fine-tuned Llama 405B on AMD GPUs by CuriouslyC on Monday September 23, 2024 @07:15PM (#41631587)
TPUs are slow but efficient and AMD has bugs but for some things works quite well. Nvidia is obviously the gold standard.
[ Parent | on HN ]
Re:We fine-tuned Llama 405B on AMD GPUs by felarof on Monday September 23, 2024 @08:34PM (#41631978)
Haha, TPUs are not slow :) All of Google's training (including Gemini models) is done on TPUs.

There are good 1p [a] and 3p [b] benchmarks comparing TPUs vs NVIDIA GPUs.

[a] - https://github.com/GoogleCloudPlatform/vertex-ai-samples/blo...

[b] - https://arxiv.org/pdf/2309.07181

[ Parent | on HN ]
Re:We fine-tuned Llama 405B on AMD GPUs by felarof on Monday September 23, 2024 @08:36PM (#41631984)
Good question. No good metric give performance depends on software stack (JAX vs PyTorch) + optimizations.

But my take performance per dollar of TPU > AMD > NVIDIA.

[ Parent | on HN ]