News for hackers. Stuff that matters.
Re:Azure announces new AI optimized VM series featuring AMD'... by kibibu on Wednesday November 15, 2023 @03:05PM (#38281697)
The article highlights that onnx, deepspeed, pytorch and tensorflow all support these AMD processors
Re:Azure announces new AI optimized VM series featuring AMD'... by CharlesW on Wednesday November 15, 2023 @03:14PM (#38281806)
Thanks, so I think you're saying that there are two related things happening that are eliminating NVIDIA's CUDA moat: (1) Developers are mostly using libraries instead of writing "to the metal" (CUDA), and (2) many popular libraries have added support for AMD GPUs. Does that capture it?
Re:Azure announces new AI optimized VM series featuring AMD'... by latchkey on Wednesday November 15, 2023 @03:25PM (#38281955)
Re:Azure announces new AI optimized VM series featuring AMD'... by brucethemoose2 on Wednesday November 15, 2023 @03:46PM (#38282213)
It's the opposite in my experience.

Pure PyTorch mostly works OK, but some libraries implementing crazy optimized, hand written kernels and such will have some trouble.

So (for instance) maybe you can run an LLM in a particular PyTorch framework, but flash attention 2 doesn't support your AMD card, so performance and memory use takes a hit.

Or maybe the library works on an Intel XPU with like 5 changed lines in the entire library (rename "cuda" to "xpu"), but no one bothered to add it, or maybe the dev doesn't even want to support the PR.

Re:Azure announces new AI optimized VM series featuring AMD'... by sevagh on Thursday November 16, 2023 @07:33AM (#38288779)
Not that I maintain any hugely popular deep learning code, but my own "hobby"/dev hesitance at adding multiple devices is that I don't own them to test them.

If I have an NVIDIA GPU, the only PyTorch backend I can promise even "somewhat works on my machine" is CUDA.

Re:Azure announces new AI optimized VM series featuring AMD'... by tails4e on Wednesday November 15, 2023 @05:42PM (#38283640)
AMD also have HIP which is almost exactly CUDA, but target agnostic, so basically now CUDA code will now run on AMD hardware. So they have the low level via HIP, and have also integrated into the libraries like pytorch. It's a great bit chip out of that moat.