News for hackers. Stuff that matters.
Re:Azure announces new AI optimized VM series featuring AMD'... by CharlesW on Wednesday November 15, 2023 @02:59PM (#38281599)
I could've sworn that it was once a given that people doing serious AI work needed CUDA-capable GPUs. Is that no longer the case?
Re:Azure announces new AI optimized VM series featuring AMD'... by brucethemoose2 on Wednesday November 15, 2023 @03:36PM (#38282095)
AMD is "CUDA compatible" in theory.

Intel is taking a slightly different approach, and is going for "PyTorch compatible."

You will hear endless negative anecdotes about ROCm/OpenVINO, but they both do seem to be getting better with each update.

Re:Azure announces new AI optimized VM series featuring AMD'... by kibibu on Wednesday November 15, 2023 @03:05PM (#38281697)
The article highlights that onnx, deepspeed, pytorch and tensorflow all support these AMD processors
Re:Azure announces new AI optimized VM series featuring AMD'... by CharlesW on Wednesday November 15, 2023 @03:14PM (#38281806)
Thanks, so I think you're saying that there are two related things happening that are eliminating NVIDIA's CUDA moat: (1) Developers are mostly using libraries instead of writing "to the metal" (CUDA), and (2) many popular libraries have added support for AMD GPUs. Does that capture it?
Re:Azure announces new AI optimized VM series featuring AMD'... by latchkey on Wednesday November 15, 2023 @03:25PM (#38281955)
Re:Azure announces new AI optimized VM series featuring AMD'... by brucethemoose2 on Wednesday November 15, 2023 @03:46PM (#38282213)
It's the opposite in my experience.

Pure PyTorch mostly works OK, but some libraries implementing crazy optimized, hand written kernels and such will have some trouble.

So (for instance) maybe you can run an LLM in a particular PyTorch framework, but flash attention 2 doesn't support your AMD card, so performance and memory use takes a hit.

Or maybe the library works on an Intel XPU with like 5 changed lines in the entire library (rename "cuda" to "xpu"), but no one bothered to add it, or maybe the dev doesn't even want to support the PR.

Re:Azure announces new AI optimized VM series featuring AMD'... by sevagh on Thursday November 16, 2023 @07:33AM (#38288779)
Not that I maintain any hugely popular deep learning code, but my own "hobby"/dev hesitance at adding multiple devices is that I don't own them to test them.

If I have an NVIDIA GPU, the only PyTorch backend I can promise even "somewhat works on my machine" is CUDA.

Re:Azure announces new AI optimized VM series featuring AMD'... by tails4e on Wednesday November 15, 2023 @05:42PM (#38283640)
AMD also have HIP which is almost exactly CUDA, but target agnostic, so basically now CUDA code will now run on AMD hardware. So they have the low level via HIP, and have also integrated into the libraries like pytorch. It's a great bit chip out of that moat.
Re:Azure announces new AI optimized VM series featuring AMD'... by modeless on Wednesday November 15, 2023 @03:16PM (#38281823)
Now that AI made Nvidia a trillion dollar company AMD has finally woken up and realized what they should have ten years ago: that they need to invest in the software side of things more. There is movement now, and you can actually do some AI things on AMD hardware, but it will take a long time for them to catch up to Nvidia.
Re:Azure announces new AI optimized VM series featuring AMD'... by latchkey on Wednesday November 15, 2023 @03:24PM (#38281945)
AMD beat Intel in the server CPU market.
Re:Azure announces new AI optimized VM series featuring AMD'... by modeless on Wednesday November 15, 2023 @06:22PM (#38283857)
AMD and Apple both have TSMC to thank for their ability to make chips that beat Intel's.
Re:Azure announces new AI optimized VM series featuring AMD'... by latchkey on Wednesday November 15, 2023 @07:45PM (#38284517)
Re:Azure announces new AI optimized VM series featuring AMD'... by malfist on Wednesday November 15, 2023 @09:31PM (#38285263)
That's a bold assertion with no evidence to back it up
Re:Azure announces new AI optimized VM series featuring AMD'... by latchkey on Thursday November 16, 2023 @12:38AM (#38286128)
I've been running large scale data centers for years now with tens of thousands of machines. I'd personally never put an Intel cpu into something that I cared about. We used $30 cheapo intel cpus for our mining rigs cause they literally didn't need anything more than that. But, anything else we buy for servers, is always going to be AMD. One caveat to that would be if we have some customer that demanded Intel and we'd lose the business otherwise.

So yea, in my eyes, AMD beat Intel. I don't think I need any more evidence than that. ¯\_(ツ)_/¯

Re:Azure announces new AI optimized VM series featuring AMD'... by malfist on Thursday November 16, 2023 @09:38PM (#38298671)
There's a big difference between AMD beating Intel in your eyes vs the market.
Re:Azure announces new AI optimized VM series featuring AMD'... by latchkey on Friday November 17, 2023 @05:02PM (#38311101)
¯\_(ツ)_/¯
Re:Azure announces new AI optimized VM series featuring AMD'... by noxa on Wednesday November 15, 2023 @03:29PM (#38281989)
challenge accepted ;)

(hello!)

Re:Azure announces new AI optimized VM series featuring AMD'... by beebeepka on Wednesday November 15, 2023 @05:57PM (#38283740)
You seem to have forgotten where AMD was 10 years ago. Company was circling the drain but they should have invested heavily in the software stack for hardware they could barely afford to design? Brilliant strategy. Now's the time to invest because they finally have the resources. The ai train is not going anywhere anytime soon
Re:Azure announces new AI optimized VM series featuring AMD'... by modeless on Wednesday November 15, 2023 @06:24PM (#38283879)
It would have paid for itself a hundred times over. And if they had started ten years ago they wouldn't have been far behind and the required investment would have initially been much less.
Re:Azure announces new AI optimized VM series featuring AMD'... by hypercube33 on Thursday November 16, 2023 @05:39AM (#38287931)
Ten years ago they were valued around $1 a share and teetering on bankruptcy. If they didn't ship Ryzen there would be no AMD today at least not what it was or is.

There are a lot of moving parts around all of this. AMD was still dealing with their fab breakup and fallout.

They basically cut all the fat and some muscle to keep pushing forward so I agree with the other poster that now is the time to focus on software and growth.

Re:Azure announces new AI optimized VM series featuring AMD'... by sevagh on Thursday November 16, 2023 @07:34AM (#38288794)
If AMD 10 years ago tried to compete with NVIDIA and Intel both simultaneously, would it have survived? They did well to stay in business, and now they can address the AI thing.
Re:Azure announces new AI optimized VM series featuring AMD'... by faeriechangling on Wednesday November 15, 2023 @03:12PM (#38281768)
There aren't enough CUDA-capable GPUs to buy. So people will enter learn how to use products like the MI300X or they won't have AI.
Re:Azure announces new AI optimized VM series featuring AMD'... by epolanski on Wednesday November 15, 2023 @03:26PM (#38281962)
It's also a matter of price. Nvidia sells their top GPUs at insane premiums. For Microsoft which is such a huge player and the major AI vendor and owns a lot of OpenAI they have no point into cornering themselves into Nvidia dependence.

They benefit from competition, not from bending to one vendor.

Re:Azure announces new AI optimized VM series featuring AMD'... by latchkey on Wednesday November 15, 2023 @05:35PM (#38283582)
> Nvidia sells their top GPUs at insane premiums.

It is all sold out, for years. You can't sell something you don't have.

https://www.theregister.com/2023/09/08/tsmc_ai_chip_crunch/

Re:Azure announces new AI optimized VM series featuring AMD'... by treprinum on Wednesday November 15, 2023 @03:20PM (#38281899)
That's the joke. It still is. ROCm is nowhere near production-ready and if MS thinks devs will want to waste their time on random errors then good luck with that business. AMD cards are also super expensive so not sure what their competitiveness is supposed to be?
Re:Azure announces new AI optimized VM series featuring AMD'... by latchkey on Wednesday November 15, 2023 @03:22PM (#38281921)
Expensive is not entirely relevant. They are available (for now).

https://twitter.com/sama/status/1724626002595471740

ROCm has also made a lot of advances in recent times.

https://www.databricks.com/blog/training-llms-scale-amd-mi25...

Re:Azure announces new AI optimized VM series featuring AMD'... by kristianp on Wednesday November 15, 2023 @05:25PM (#38283480)
Thread about that tweet (or should I say xeet) from SamA: https://news.ycombinator.com/item?id=38274427
Re:Azure announces new AI optimized VM series featuring AMD'... by epolanski on Wednesday November 15, 2023 @03:23PM (#38281936)
Microsoft can develop their own software. We are talking a company that places billions in orders to Nvidia, they can afford to develop the tools and obviously AMD will throw millions and millions in support to court such a customer.
Re:Azure announces new AI optimized VM series featuring AMD'... by brucethemoose2 on Wednesday November 15, 2023 @03:53PM (#38282326)
The MI300 is a massive GPU compared to even the new H200.

It was designed as a combined CPU/GPU for supercomputers, with shared memory. But then the AI craze hit, so AMD spun a variant into a pure GPU AI accelerator real quick, which they could actually pull off because the GPU silicon is modular.

...So thats why it cost a fortune. Its really a jury rigged HPC product.

Re:Azure announces new AI optimized VM series featuring AMD'... by pclmulqdq on Wednesday November 15, 2023 @06:41PM (#38283987)
The H100 is, to some degree, also a jury rigged HPC product. The H200 is starting to reach a more "AI" balance between compute and memory bandwidth.
Re:Azure announces new AI optimized VM series featuring AMD'... by treprinum on Wednesday November 15, 2023 @07:41PM (#38284482)
I can see MI300 being used for CFD, nuclear physics simulation, cloud GPU virtualization etc. but for AI it's simply not there regardless of AMD's and its affiliated startups' marketing.
Re:Azure announces new AI optimized VM series featuring AMD'... by latchkey on Wednesday November 15, 2023 @08:00PM (#38284640)
You're thinking the MI250. MI300x is a beast for AI.
Re:Azure announces new AI optimized VM series featuring AMD'... by treprinum on Thursday November 16, 2023 @11:01AM (#38291166)
Vega was an ML monster on paper as well, handily beating 1080Ti. Then the reality set in...
Re:Azure announces new AI optimized VM series featuring AMD'... by latchkey on Thursday November 16, 2023 @11:04AM (#38291192)
These aren’t consumer cards. Apples and oranges.
Re:Azure announces new AI optimized VM series featuring AMD'... by qeternity on Wednesday November 15, 2023 @03:05PM (#38281688)
Necessity is the mother of invention.
Re:Azure announces new AI optimized VM series featuring AMD'... by yieldcrv on Wednesday November 15, 2023 @08:43PM (#38284948)
ROCm support for a lot of AI processes fixed that