So basically, I'm betting that AMD has had (or is having) a change of heart and is genuinely committed to AI/ML on their products. Time may ultimately prove me wrong, but so be it if that proves to be the case.
And FWIW, one reason for my optimism is that, whatever you think about the state of ROCm today, they are clearly investing heavily into the platform and continually working on it. You can see that just from looking at the activity on their Github repos:
https://github.com/orgs/ROCmSoftwarePlatform/repositories
There is constant activity and has been for some time, which I take as a good sign. Yes, it's just one signal among many that one could consider, but I think it's an important one.
My example about LLMs was just to show that AMD is simply not part of the conversation. Three month before you could have made the same point about another approach.
And still, if you'd actually have to risk money, you and me both know you'd never invest in AMD hardware for AI or start developing on it something high stakes.
I mean, look at geohot he tried and just gave up entirely and AMD.
I assume you meant "you" in the royal sense. Since I personally am, in point of fact, investing in AMD hardware for AI. Yeah, it's a gamble, but that's my style. And I have to admit, part of it is driven by ideological reasons (ROCm being open source) and a simple desire to support AMD since I want them to become a serious competitor to NVIDIA. I believe that's an outcome that would benefit everybody.
I mean, look at geohot he tried and just gave up entirely and AMD.
I have to admit, I don't find that particularly compelling in any regard. Nothing against geohot, he's clearly a smart dude, but... I'm not judging a company based on his interactions with them. shrug
So for now, I don't see AMD getting any traction. And apparently, the quality of the drivers has not been improving. Time will tell.
Beyond that, it's just going to be wherever the research takes me. And not everything is necessarily going to be particularly suited for running on ROCm. I'll use CUDA on NVIDIA hardware as and when needed, and I'm also open to dabbling in low-level hardware stuff and playing around with DSP's, analog computing, FPGA's, etc. One thing I want to work with a bit is some Spiking Neural Network stuff, neuromorphic approaches, etc. So not sure where this will end up.
Heck, it's possible that some of the stuff I work with will turn out to be well suited for running on a plain old CPU, which is one reason I spec's a relatively high-end Ryzen CPU and 64GB of system RAM for these machines, which otherwise wouldn't necessarily be all that important for "pure" GPU computation.
Maybe I'll even wind up going "old school" and building myself a Beowulf cluster! So I can finally answer the age old question about "imagine a Beowulf cluster of these..."
NVIDIA is about to walk off with a trillion dollars because nobody at AMD “gets it”.
With no meaningful competition, NVIDIA will gouge as hard as they can. Such as charging $50K for a card that’s not too different to a 4090 but with more memory.
I think they "get it" OK. Whether or not they can formulate a viable strategy and execute it is one question, but they get the idea that "AI is important" and they know where they stand vis-a-vis NVIDIA.
This is another reason I'm willing to invest some time and money in working with AMD products for AI/ML. History has shown us their ability to go toe-to-toe with a seemingly unassailable industry titan before, and they came out in pretty good shape then.
https://www.forbes.com/sites/iainmartin/2023/05/31/lisa-su-s...
ROCm is getting there, slowly.
There's a language called HIP which is a fairly close approximation to cuda. You can probably convert one to the other with determination and regex. The GPUs themselves are fundamentally different in ways that hopefully don't matter to your application (warp synchronisation is the big one in my opinion, but I suspect cuda applications ignore it and just live with the race conditions).
I mean, sure, you could compile your stuff to XLA or just, I dunno, set up 800 of these cards and train the whole thing on ROCM. But then would you really, really use AMD instead of some TPUs?
AMD made Machine Learning either impossible, unsupported or a chore on most of their hardware for years. Their stack sucks and no one wanted to implement it, and in fact they rarely supported most of their own GPUs.
Yes you have GPUs. But we need also drivers. And software. People have been yelling at AMD about this for years and years.
Instead, AMD has made it clear that Machine Learning is not a priority for the company. Hence, they deserve the lack of traction. Investing in AMD hardware for ML has literally been a mistake at every point in recent history. Imagine if you bought a bunch of (insert last Gen card which is no longer supported by their stack) how dumb you'd look.
Releasing a GPGPU card now? Honestly, why bother? No one is gonna buy it.