I got a Stable Diffusion thing up and running locally on a relatively old AMD RX 580 with 8GB of memory, and default settings of 512x512 with 10 (I think) passes, and it takes about 10 minutes to generate an image. On a similarly aged Nvidia 1060 with 6GB of memory that a colleague of mine uses for their local Stable Diffusion instance, it takes about 20 seconds to generate an image.

I don't know how apples and oranges this comparison is, my OS is Linux and his is Windows, for example, but 10 minutes versus 20 seconds is, for me, a massively disappointing difference.

I'm sure the gap has decreased in the intervening 6-7 years, but I don't know how much - such that I'd be opting for a second hand Nvidia over a new AMD (and I philosophically prefer AMD).

This may be a worthwhile read if you're looking to squeeze the most performance: https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...

It's still a pain in other ways. AMD does not support ROCm on consumer-grade GPUs, it only "happens to work". I had to explicitly install the 3.10 python interpreter and use an older torch version, because seemingly, the newer ROCm releases do not support the RX5700 XT. Also ROCm only works on Linux.

My experience with a RX6800 was that compiling ROCm for myself for use with Pytorch just worked, though from reading online it's clear that's a rarity. Current gen cards apparently don't even work.