News for hackers. Stuff that matters.
Re:AMD's AI Future Is Rack Scale 'Helios' by halJordan on Saturday June 14, 2025 @10:13PM (#44280094)
Honestly that was a hard read. I hope that guy gets an mi355 just for writing this.

AMD deserves exactly zero of the credulity this writer heaps onto them. They just spent four months not supporting their rdna4 lineup in rocm after launch. AMD is functionally capable of day120 support. None of the benchmarks disambiguated where the performance is coming from. 100% they are lying on some level, representing their fp4 performance against fp 8/16.

Re:AMD's AI Future Is Rack Scale 'Helios' by jchw on Sunday June 15, 2025 @12:35AM (#44280561)
I still find their delay with properly investing in ROCm on client to be rather shocking, but in fairness they did finally announce that they would be supporting client cards on day 1[1]. Of course, AMD has to keep the promise for it to matter, but they really do seem to, for whatever reason, finally realized just how important it is that ROCm is well-supported across their entire stack (among many other investments they've announced recently.)

It's baffling that AMD is the same company that makes both Ryzen and Radeon, but the year-to-date for Radeon has been very good, aside from the official ROCm support for RDNA4 taking far too long. I wouldn't get overly optimistic; even if AMD finally committed hard to ROCm and Radeon it doesn't mean they'll be able to compete effectively against NVIDIA, but the consumer showing wasn't so bad so far with the 9070 XT and FSR4, so I'm cautiously optimistic they've decided to try to miss some opportunities to miss opportunities. Let's see how long these promises last... Maybe longer than a Threadripper socket, if we're lucky :)

[1]: https://www.phoronix.com/news/AMD-ROCm-H2-2025

Re:AMD's AI Future Is Rack Scale 'Helios' by roenxi on Sunday June 15, 2025 @01:47AM (#44280797)
Is this day 1 support a claim about the future or something they've demonstrated? Because if it involves the future it is safer to just assume AMD will muck it up somehow when it comes to their AI chips. It isn't like their failure in the space is a weird one-off - it has been confusingly systemic for years. It'd be nice if they pull it off, but it could easily be day 1 support for a chip that turns out to crash the computer.

I dunno; I suppose they can execute on server parts. But regardless, a good plan here is to let someone else go first and report back.

Re:AMD's AI Future Is Rack Scale 'Helios' by jchw on Sunday June 15, 2025 @07:31AM (#44281770)
They've been able to execute well for Ryzen, EPYC, and Radeon in the data center. I don't really think there's any reason to believe they can't or even wouldn't be able to do ROCm on client cards, but up until recently they wouldn't commit.
Re:AMD's AI Future Is Rack Scale 'Helios' by pclmulqdq on Saturday June 14, 2025 @10:24PM (#44280127)
AMD doesn't care about you being able to do computing on their consumer GPUs. The datacenter GPUs have a pretty good software stack and great support.
Re:AMD's AI Future Is Rack Scale 'Helios' by fc417fc802 on Saturday June 14, 2025 @11:29PM (#44280322)
I'm inclined to believe it but that difference is exactly how nvidia got so far ahead of them in this space. They've consistently gone out of their way to put their GPGPU hardware and software in the hands of the average student and professional and the results speak for themselves.
Re:AMD's AI Future Is Rack Scale 'Helios' by zombiwoof on Sunday June 15, 2025 @01:51AM (#44280809)
Just look at the disaster of rocm or you need to spend 300k on software engineers to get anything so work
Re:AMD's AI Future Is Rack Scale 'Helios' by tormeh on Sunday June 15, 2025 @08:46AM (#44282038)
I wouldn't say so. Nvidia bet on machine learning a decade or so before AMD got the memo. That was a good bet on Nvidia's part. In 2015 you just had to have an Nvidia card if you wanted to do ML research. Sure, Nvidia did hand them out in some cases, but even if you bought an AMD card it just wouldn't work. It was Nvidia or go home. Even if AMD now did everything right (and they don't), there's a decade+ of momentum in Nvidia's favor.
Re:AMD's AI Future Is Rack Scale 'Helios' by stingraycharles on Sunday June 15, 2025 @12:39AM (#44280569)
Yes but then they fail to understand a lot of “long tail” home projects, opensource stuff etc is done on consumer GPUs at home, which is tremendously important for ecosystem support.
Re:AMD's AI Future Is Rack Scale 'Helios' by wmf on Sunday June 15, 2025 @01:08AM (#44280684)
What if they understand that and they don't care? Getting one hyperscaler as a customer is worth more than the entire long tail.
Re:AMD's AI Future Is Rack Scale 'Helios' by stingraycharles on Sunday June 15, 2025 @01:51AM (#44280810)
The problem is that this is short-term thinking. You need students and professionals playing around with your tools at home and/or on their work computers to drive hyperscale demand in the long term.

This is why it’s so important AMD gets their act together quickly, as the benefits of these kind of things are measured in years, not months.

Re:AMD's AI Future Is Rack Scale 'Helios' by selectodude on Sunday June 15, 2025 @01:18AM (#44280711)
Then they’re fools. Every AI maestro knows CUDA because they learned it at home.
Re:AMD's AI Future Is Rack Scale 'Helios' by jiggawatts on Sunday June 15, 2025 @01:54AM (#44280816)
It’s the same reason there’s orders of magnitude more code written for Linux than for mainframes.
Re:AMD's AI Future Is Rack Scale 'Helios' by danielheath on Sunday June 15, 2025 @02:55AM (#44280976)
Why would a hyperscaler pick the technology that’s harder to hire for (because there’s no hobbyist-to-expert pipeline)?
Re:AMD's AI Future Is Rack Scale 'Helios' by moffkalast on Sunday June 15, 2025 @03:33AM (#44281103)
Then they will stay irrelevant in the GPU space like they have been so far.
Re:AMD's AI Future Is Rack Scale 'Helios' by lhl on Sunday June 15, 2025 @04:19AM (#44281211)
On the corp side you have FB w/ PyTorch, xformers (still pretty iffy on AMD support tbt) and MS w/ DeepSpeed. But let's see about some others:

Flash Attention: academia, 2y behind for AMD support

bitsandbytes: academia, 2y behind for AMD support

Marlin: academia, no AMD support

FlashInfer: acadedmia/startup, no AMD

ThunderKittens: academia, no AMD support

DeepGEMM, DeepEP, FlashMLA: ofc, nothing from China supports AMD

Without the long tail AMD will continue to always be in a position where they have to scramble to try to add second tier support years later themselves, while Nvidia continues to get all the latest and greatest for free.

This is just off the top of my head on the LLM side where I'm focused on, btw. Whenever I look at image/video it's even more grim.

Re:AMD's AI Future Is Rack Scale 'Helios' by jimmySixDOF on Sunday June 15, 2025 @04:33AM (#44281247)
Modular says Max/Mojo will change this and make refactoring between different vendors (and different lines of the same vendor) less of a showstopper but tbd for now
Re:AMD's AI Future Is Rack Scale 'Helios' by pjmlp on Sunday June 15, 2025 @12:45PM (#44283308)
The judge is still out there regarding if Max/Mojo is going to be something that the large majority cares about.
Re:AMD's AI Future Is Rack Scale 'Helios' by littlestymaar on Sunday June 15, 2025 @04:36AM (#44281257)
Why should we care about them if they don't care?

I mean of they want to stay at a fraction of the market value and profit of their direct competitor, good for them.

Re:AMD's AI Future Is Rack Scale 'Helios' by dummydummy1234 on Sunday June 15, 2025 @07:47AM (#44281830)
I want a competitive market so I can have cheaper gpus.

It's Nvidia, AMD, and maybe Intel.

Re:AMD's AI Future Is Rack Scale 'Helios' by cma on Sunday June 15, 2025 @12:47AM (#44280604)
Nvidia started removing nvlink with the 4000 series, they aren't heavily focused on it either anymore and want to sell the workstation cards for uses like training models at home.
Re:AMD's AI Future Is Rack Scale 'Helios' by 7speter on Tuesday June 17, 2025 @12:17AM (#44295718)
As a beginner, you can get a lot done on a 4060ti/5060ti 16gb, much less a 4090/5090.

Heck I’ve been able to work through the early chapters of the FastAI book using a lowly Quadro p1000

Re:AMD's AI Future Is Rack Scale 'Helios' by viewtransform on Saturday June 14, 2025 @11:30PM (#44280325)
AMD is offering AMD Developer Cloud (https://www.amd.com/en/blogs/2025/introducing-the-amd-develo...)

"25 complimentary GPU hours (approximately $50 US of credit for a single MI300X GPU instance), available for 10 days. If you need additional hours, we've made it easy to request additional credits."

Re:AMD's AI Future Is Rack Scale 'Helios' by archerx on Sunday June 15, 2025 @12:13AM (#44280474)
If they care about their future they should. I am a die hard AMD supporter and even I am getting over their mediocrity and what seems to be constant self sabotage in the GPU department.
Re:AMD's AI Future Is Rack Scale 'Helios' by zombiwoof on Sunday June 15, 2025 @01:53AM (#44280813)
It’s the AMD management . They just are recycling 20 year VP lifers at AMD to take over key projects
Re:AMD's AI Future Is Rack Scale 'Helios' by archerx on Sunday June 15, 2025 @09:09AM (#44282127)
They could have slapped 48gb of vram on their new Radeon cards and they would have instantly sold out but that would cut into cousins profit margin at nvidia so that’s obviously a no go.
Re:AMD's AI Future Is Rack Scale 'Helios' by booder1 on Sunday June 15, 2025 @12:13AM (#44280477)
I have had trained on both large AMD and Nvidia clusters and your right AMD support is good. I never had to talk to Nvidia support. That was better.

They should care about the availability of their hardware so large customers don't have to find and fix their bugs. Let consumers do that...

Re:AMD's AI Future Is Rack Scale 'Helios' by echelon on Sunday June 15, 2025 @12:19AM (#44280499)
> AMD doesn't care about you being able to do computing on their consumer GPUs

Makes it a little hard to develop for without consumer GPU support...

Re:AMD's AI Future Is Rack Scale 'Helios' by fooker on Sunday June 15, 2025 @02:23AM (#44280879)
It’s the same software stack.
Re:AMD's AI Future Is Rack Scale 'Helios' by caycep on Saturday June 14, 2025 @10:34PM (#44280150)
this is ROCm?
Re:AMD's AI Future Is Rack Scale 'Helios' by fooblaster on Saturday June 14, 2025 @10:39PM (#44280174)
Yes, the mi300x/mi250 are best supported as they directly compete with data center gpus from Nvidia which actually make money. Desktop is a rounding error by comparison.
Re:AMD's AI Future Is Rack Scale 'Helios' by shmerl on Sunday June 15, 2025 @12:53AM (#44280628)
Aren't they addressing it with the unified UDNA architecture? That's going to be a thing in the future GPUs, making consumer and datacenter ones share the same arch.

Different architectures was probably a big reason for the above issue.

Re:AMD's AI Future Is Rack Scale 'Helios' by pjmlp on Sunday June 15, 2025 @12:44PM (#44283297)
Except they forget people get to adopt technologies by learning them on their consumer hardware.
Re:AMD's AI Future Is Rack Scale 'Helios' by zombiwoof on Sunday June 15, 2025 @01:50AM (#44280806)
Exactly.

AMD is a marketing company now

Re:AMD's AI Future Is Rack Scale 'Helios' by ethbr1 on Sunday June 15, 2025 @05:41AM (#44281430)
> I hope that guy gets an mi355 just for writing this. AMD deserves exactly zero of the credulity this writer heaps onto them.

You mean Ryan Smith of late AnandTech fame?

https://www.anandtech.com/author/85/