This is a pet peeve of mine, but A1111 is not a great example of a ML project. The backend codebase is a mess compared to diffusers, its slow even on Nvidia, its history is filled with controversy (including license breaking and outright piracy in the codebase), the frontend is buggy and unstable. I appreciate how it was a pioneer in 2022-H1 2023, but I am somewhat annoyed its still the de facto SD backend.

Better maintained inference projects are picking up considerable AMD/Intel support. Sometimes even Apple Silicon support.

The only real competitor is ComfyUI, because without the huge ecosystem around your other front end, you lose MASSIVE capabilities, for example, losing this is a big mistake: https://github.com/ljleb/prompt-fusion-extension

Is there something like that extension for comfy yet?

At least on Nvidia, A1111 has had near identical performance to ComfyUI on SDXL and SD1.5 for me. Besides that, there were typical minor problems that happen when some guy's hobby project goes viral and suddenly is under massive scrutiny. It was the first "good enough" interface and now there is massive community inertia with extensions, plugins, etc.

ComfyUI has found a place in the image pipeline design niche, but A1111 still seems like the best choice for new users who just want to spin up Stable Diffusion and generate some art for a D&D campaign or whatever.

Thats because ComfyUI is also based on the old SAI backend, not diffusers like, say, VoltaML.

> but if they can't get developer communities to build around their equivalent to CUDA and its ecosystem than it's not looking rosy for AMDs future.

Dec 6th was an announcement from Lisa Su about the MI300x. It also included a full on commitment to ROCm as well as other projects like pyTorch. They are clearly aware of what you're talking about, and working on it. It isn't something that can magically be fixed over night, but it is something that people can complain about on HN over and over and over again...

Some of us have been complaining for close to a decade and there's been multiple promises that have gone no where. It's better to keep complaining until the problem is actually fixed. CUDA is 16 years old and AMD has been "aware" of the issue for at least 10.

How could they promise to fix something that didn't exist 10-16 years ago?

In my eyes, the incentives weren't there before. But, they sure are now!

OpenCL did exist 10 years ago! Rocm has been around for 7 years. Pytorch is 7 years old and had CUDA support on day one and still doesn't have RocM support. Before the LLM craze people still wanted an alternative to nvidia and could see the writing on the wall. There was a bunch of complaining about Rocm and OpenCL didn't deliver. GeoHotz tried to get a RocM backend working and the examples just didn't work.

You can't build a software platform by just willing it into existence. It means sitting down with developers, writing documentation and support users. It means tackling use cases other than llama so that users feel confident in building the next greatest thing on your platform. At it is today, AMD will always be playing catch up and people will continue to push nvidia because the latest and greatest will be on Cuda.

I guess for those of spending money on real gpus for the last 10 years, nvidia was incentivized to take our money but amd wasn't.

Before the current crazy of AI, what else generated large sums of money for GPUs?

Games, rendering and crypto.

If I was a hardware company, that's all that I would have focused on. Certainly AMD could have done better on the software/developer front, nobody is arguing that.

I tend to live in the present and what matters is what they are saying and doing today. You can choose to be pissed off about the past, or you can work towards the future.

And nvidia is killing it right now because they predicted the need _before_ the incentives were there.

Nobody is arguing that AMD didn't get caught with their pants around their ankles or that they have a great reputation with developers. What matters is their statements and more importantly, actions, moving forward.

Right now, they are cranking out ROCm releases on schedule, contributing to a bunch of AI open source projects, listening to crazy people like George Hotz and have released a GPU product that leapfrogs the H100 and is built on OAM-UBB, at 1/3 the FLOPS/$, while also promoting fast ethernet standards.

I'd call that a pretty good start.

Which makes it even more stupid that AMD does not at least try to compensate for their shortcoming by selling large memory GPUs, there would be a reason to develop for team red

Yeah.

If there was a 40GB-48GB 7900 SKU, I would have bought it over a 3090 in a heartbeat, even at a big premium over the 24GB card. And I'd be debugging ROCm support in ML projects left and right.

But I am not. The 24GB SKU is just not worth the trouble over a 3090, which is more RAM efficient out of the box. I can't pay $5K for a W7900.