Moving target, honestly just get PyTorch working fully (loads of stuff just doesn’t work on AMD hardware) and also make it work on all graphics cards from a certain generation. The matrix of support needed GFX cards, architectures and software together is quite astounding but still yes that should have at least that working and equivalent custom kernels.

That would be a great start.

The headline that PyTorch has full compatibility on all AMD GPUs would increase their stock by > $50 billion overnight. They should do it even if it takes 500 engineers and 2 years.

Does anybody really understand why this hasn‘t been done? I know about ongoing efforts but is it really THAT difficult?

You know it’s probably a combination of things but mostly that AMD do not have a capable software team… probably not the individuals but the managers likely don’t have a clue.

Aaaaaaand torch is not a simple easy target. You don't just want support but high-performance optimized support on a pretty-complex moving target... maybe better/easier than CUDA but not that much it seems.

AMD did hire someone to do this and IIRC he did, but they were afraid of Nvidia lawyers and he released it outside of the company?

Surely they could hire some good lawyers if that means they make billions upon billions? AFAIK there's nothing illegal about creating compatibility layers. Otherwise WINE would have shut down long ago.

Depends on what code they wrote. If they used LLMs to write it, it could contain proprietary nvidia parts. Someone would then have to review that, but can't, because maybe the nvidia code that came from the LLM isn't even public.

So the strategy to publish independently, wait and see if nvidia lawyers have anything to say about it, would be a very smart move.

The Oracle case was about the stubs of the API being considered copyrighted I believe. The argument wasn't that Google used any of their code, it was that by using the same functional names they were making a thought crime.

The last time something similar happened (Google vs Oracle), the legal battles lasted more than a decade. It would be a very bold decision by ARM to commit to this strategy (implement "CUDA" and fight it out in courts).

That means Google got to use Java for a decade. A decade-long legal battle is great news for whoever seems to be in the wrong, as long as they can still afford lawyers. Remember, they don't claw back dividends or anything.

Just allow me to doubt that one (1) programmer is all AMD would need to close up the software gap to NVIDIA...

Are you suggesting that CUDA is the entirety of the "software gap", because it's a lot more than that. That seems like a strawman argument.

Andrzej Janik.

Starter at Intel working on it, they passed because there was no business there.

AMD picked it up and funded it from 2022. They stopped in 2024, but his contract allowed the release of the software in such an event.

Now it's ZLUDA.

They've been doing it all the time and it's called HIP. Nowadays it works pretty well on a few supported GPUs (CDNA 3 and RDNA 4).

it's astounding to me how many people pop off about "AMD SHOULD SUPPORT CUDA" not knowing that HIP (and hipify) has been around for literally a decade now.

Please explain to me why all the major players are buying Nvidia then? Is HIP a drop in replacement? No.

You have to port every piece of software you want to use. It's ridiculous to call this a solution.

Major players in China don't play like that. MooreThreads, Lisuan, and many other smaller companies all have their own porting kits, which are basically copied from HIP. They just port every piece of software and it just works.

If you want to fight against Nvidia monopoly, then don't just rant, but buy a GPU other than Nvidia and build on it. Check my GitHub and you'll see what I'm doing.

> Is HIP a drop in replacement? No.

You don't understand what HIP is - HIP is AMD's runtime API. it resembles CUDA runtime APIs but it's not the same thing and it doesn't need to be - the hard part of porting CUDA isn't the runtime APIs. hipify is the thing that translates both runtime and kernels. Now is hipify a drop-in replacement? No of course but because the two vendors have different architectures. So it's absolutely laughable to imagine that some random could come anywhere near "drop-in replacement" when AMD can't (again: because of fundamental architecture differences).

Who said "some random"? Read the whole thread. I was suggesting AMD invest BILLIONS to make this happen. You're aguing with a straw man.

I think you misunderstand what's fundamentally possible with AMD's architecture. They can't wave a magic wand for a CUDA compatibility layer any better than Apple or Qualcomm can, it's not low-hanging fruit like DirectX or Win32 translation. Investing billions into translating CUDA on raster GPUs is a dead end.

AMD's best option is a greenfield GPU architecture that puts CUDA in the crosshairs, which is what they already did for datacenter customers with AMD Instinct.

This is a big part of AMD still not having a proper foothold in the space: AMD Instinct is quite different from what regular folks can easily put in their workstation. In Nvidia-land I can put anything from mid-range gaming cards, over a 5090 to an RTX 6000 Pro in my machine and be confident that my CUDA code will scale somewhat acceptably to a datacenter GPU.

This is where I feel like Khronos could contribute, making a Compute Capability-equivalent hardware standard for vendors to implement. CUDA's versioning of hardware capabilities plays a huge role in clarifying the support matrix.

...but that requires buy-in from the rest of the industry, and it's doubtful FAANG is willing to thread that needle together. Nvidia's hedged bet against industry-wide cooperation is making Jensen the 21st century Mansa Musa.

I do not misunderstand.

Let's say you put 50-100 seasoned devs on the problem, and within 2-3 years, probably get ZLUDA to the point where most mainstream CUDA applications — ML training/inference, scientific computing, rendering — run correctly on AMD hardware at 70-80% of the performance you'd get from a native ROCm port. Even if its not optimal due to hardware differences, it would be genuinely transformative and commercially valuable.

This would give them runway for their parallel effort to build native greenfield libraries and toolkits and get adoption, and perhaps make some tweaks to future hardware iterations that make compatibility easier.

Before the "ZLUDA" project completion, they would be facing a lawsuit for IP infringement, since CUDA is owned by NVIDIA.

They would win, compatibility layers are not illegal.

Win against who? AMD is the one that asked them to take it down: https://www.tomshardware.com/pc-components/gpus/amd-asks-dev...

And while compatibility layers aren't illegal, they ordinarily have to be a cleanroom design. If AMD knew that the ZLUDA dev was decompiling CUDA drivers to reverse-engineer a translation layer, then legally they would be on very thin ice.

ROCm is supported by the minority of AMD GPUs, and is accelerated inconsistently across GPU models. 70-80% of ROCm's performance is an unclear target, to the point that a native ROCm port would be a more transparent choice for most projects. And even then, you'll still be outperformed by CUDA the moment tensor or convolution ops are called.

Those billions are much better-off being spent on new hardware designs, and ROCm integrations with preexisting projects that make sense. Translating CUDA to AMD hardware would only advertise why Nvidia is worth so much.

> it would be genuinely transformative and commercially valuable.

Bullshit. If I had a dime for every time someone told me "my favorite raster GPU will annihilate CUDA eventually!" then I could fund the next Nvidia competitor out of pocket. Apple didn't do it, Intel didn't do it, and AMD has tried three separate times and failed. This time isn't any different, there's no genuine transformation or commercial value to unlock with outdated raster-focused designs.

No I'm arguing with someone who clearly doesn't understand GPUs

> invest BILLIONS to make this happen

As I have already said twice, they already have, it's called hipify and it works as well as you'd imagine it could (ie poorly because this is a dumb idea).

Wow you're so very smart! You should tell all the llm and stablediffusion developers who had no idea it existed! /s

HIP has been dismissed for years because it was a token effort at best. Linux only until the last year or two, and even now it only supports a small number of their cards.

Meanwhile CUDA runs on damn near anything, and both Linux and Windows.

Also, have you used AMD drivers on Windows? They can't seem to write drivers or Windows software to save their lives. AMD Adrenalin is a slow, buggy mess.

Did I mention that compute performance on AMD cards was dogshit until the last generation or so of GPUs?

Please. If HIP worked so well they would be eating into Nvidia's market share.

First, it's a porting kit, not a compatibility layer, so you can't run arbitrary CUDA apps on AMD GPUs. Second, it only runs on some of their GPUs.

This absolutely does not solve the problem.

HIP is just one of many examples of how utterly incompetent AMD is at software development.

GPU drivers, Adrenalin, Windows chipset drivers...

How many generations into the Ryzen platform are they, and they still can't get USB to work properly all the time?

AMD doesn't do USB, they source the controller IP from ASMedia, who also developed most of their chipsets.

Yeah but AMD could absolutely work on a joint venture to get something like moving along solidly.