News for hackers. Stuff that matters.
Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by irusensei on Thursday January 19, 2023 @06:45AM (#34439291)
My totally user/curious 2 sats experience:

I've recently tried to play with Stable Diffusion. I have an RTX 3060, a base M1 MBP and a laptop with an AMD RX6800m.

- The RTX 3060 just works. Setup was straightforward, performance is good.

- The M1 has those neural engine thingies but it's not compatible with SD. It can run SD on CPU but if you want to make use of the NE you need coreml specific programs and models. Issue here is that it's just not the same stuff. Prompt structure is also different. It doesn't recognize weights and put a lot more emphasis on prompt order. Most of the time it seems to ignore most of what you wrote. On the positive side running stuff on the NE is very fast and doesn't seem to be taxing to the system.

- Finally the 6800M. It should be the powerhouse of the bunch considering its gaming performance ahead of the 3060. Problem is AMD toolkit kinda sucks. They have this obscure ROCm HIP stuff that acts as some kind of translation layer to the CUDA API. It's complicated, it simply doesn't work without a bunch of obscure environment variables and only works on fp32 mode, which means it uses twice amount of video RAM for the same thing. Support is iffy as they seem to only support workstation cards. Using it often throws lots of obscure compilation errors. Bugs celebrating anniversary.

To sum things up, Nvidia is far ahead of the curve in both usability and performance. Apple is trying to do its thing but it's too early yet while AMD is in a messy situation. Hope it helps.

Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by muxr on Thursday January 19, 2023 @08:42AM (#34440130)
AMD's driver is much better on Linux than Nvidia's. And ROCm HIP stuff isn't obscure. It's what's needed to bridge Nvidia's proprietary vendor lock ins.

Nvidia is literally cancer in this space.

Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by nicolaslem on Thursday January 19, 2023 @09:44AM (#34440795)
AMD display driver on Linux is much better than nvidia's but the GPGPU situation is just ridiculous outside of the Pro cards.
Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by irusensei on Thursday January 19, 2023 @10:39AM (#34441453)
> AMD's driver is much better on Linux than Nvidia's.

1. Agree AMDGPU is awesome. 2. All tests were done in Linux with the exception of coreml on MacOS.

> And ROCm HIP stuff isn't obscure.

It is from my novice perspective. While Nvidia stuff works out of the box (even on Linux) I had to install Rocm and a special version of PyTorch for rocm. Then it only kinda worked after an assortment of environment variables to set GFX version. And even then it uses as twice as much video memory as what torch uses on Nvidia, limiting what I can do.

Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by okamiueru on Thursday January 19, 2023 @10:46AM (#34441539)
When you say "out of the box", then it's distros like pop!_os? Or has it become more common to automatically install proprietary drivers? Asking out of curiosity.
Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by irusensei on Thursday January 19, 2023 @11:17AM (#34441957)
Not referring to drivers but runtime libraries.
Re:GPUs for Deep Learning in 2023 – An In-depth Analysis by rpep on Thursday January 19, 2023 @11:22AM (#34442020)
The ROCm libraries and their documentation are nowhere near as good though.

For e.g. compare rocFFT to cuFFT. They're worlds apart still.