Re:Run Stable Diffusion on Your M1 Mac’s GPU
by gzer0 on Thursday September 01, 2022 @02:26PM
(#32680820)
The difference between an M2 air (8gb/512gb) versus an M1 pro (16gb/1tb) is much more than I expected.
I knew there would be some throttling due to the m2 air's fanless model, but I had no idea it would be a 20x difference (albeit, the m1 pro does have double the RAM. I don't have any other macbooks to test this on).
[ Parent
| on HN ]
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by valley_guy_12 on Thursday September 01, 2022 @02:39PM
(#32680994)
That's probably due to swapping due to the 8GB of RAM. People who have run Stable Diffusion on M2 airs with 16 GB of RAM seem to get performance that is in line with their GPU core count.
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by bfirsh on Thursday September 01, 2022 @03:21PM
(#32681654)
Correct. We've been seeing 8GB is super slow, >=16GB is fast. We'll add that to the prerequisites.
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by andybak on Thursday September 01, 2022 @02:38PM
(#32680992)
Unscientifically that puts the M1 Pro GPU at about 25% of the performance of a RTX 3080.
Not too shabby...
EDIT - this comment implies it's much faster: https://news.ycombinator.com/item?id=32679518
If that's correct then it's close to matching my 3080 (mobile).
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by valley_guy_12 on Thursday September 01, 2022 @05:28PM
(#32683500)
It's likely that a significant fraction of the perf difference between Apple' GPUs and NVIDIA GPUs is due to NVIDIA's CUDA being high optimized, and Pytorch being tuned to work with CUDA.
If Pytorch's metal support improves and Apple's Metal drivers improve (big ifs), it's likely that Apple's GPUs will perform better relatively to NVIDIA than they currently do.
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by fassssst on Thursday September 01, 2022 @02:44PM
(#32681088)
img2img runs in 6 seconds on my GeForce 3080 12 GB. 6+ it\s depending on how much GPU memory is available. If I have any electron apps running it slows down dramatically.
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by andybak on Thursday September 01, 2022 @03:13PM
(#32681549)
Curious about:
1. Image size
2. Steps
3. What your numbers are for text2img
4. (most importantly) are you including the 30 seconds or so it takes to load the model initially? i.e. if you were to run 10 prompts and then divide the total time by 10, what are your numbers?
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by cube2222 on Thursday September 01, 2022 @04:14PM
(#32682429)
Re 4 the lstein repo gives you an interactive repl, so you don't have to reload the model on every prompt.
I also have a 3080 and as far as I remember (not at my pc right now) it was 3-10 secs for img2img 512px cfg13 50 steps batch size 1 dimm sampler.
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by fragmede on Thursday September 01, 2022 @03:07PM
(#32681447)
what args are you passing to img2img?
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by Birch-san on Friday September 02, 2022 @03:22AM
(#32687655)
Neither of these should take minutes. Try Heun sampler, 8 steps, Karras noise schedule. Should be possible to get good images in 11 secs (or 10 secs if you go down to 7 steps). measurements admittedly from M1 Max.
https://twitter.com/Birchlabs/status/1565029734865584143?s=2...
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by _ph_ on Thursday September 01, 2022 @06:43PM
(#32684334)
I would assume it is the memory. The test command from the discussed link runs in slightly over 2 minutes on my M2 Air (16gb). How long does it take for yours?
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by qayxc on Thursday September 01, 2022 @02:32PM
(#32680909)
I suspect the lack of RAM is the issue here.
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by JimmyAustin on Thursday September 01, 2022 @02:42PM
(#32681050)
I suspect that the M2 air is thrashing the disk pretty aggressively. Diffusion models rerun the same model once per step, so for a generation with 50 steps, you copy the entire model in and out of memory 50 times. That's going to kill performance.
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by bm-rf on Thursday September 01, 2022 @02:47PM
(#32681134)
I believe the model is copied into ram once upon calling StableDiffusionPipeline, unless the mac implementation partially loads the model due to only having 8G of ram.
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by schleck8 on Thursday September 01, 2022 @02:52PM
(#32681218)
It's only copied to VRAM once when implemented correctly.
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by astrange on Thursday September 01, 2022 @05:36PM
(#32683608)
M1 is a unified memory system and doesn't have VRAM.
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by schleck8 on Thursday September 01, 2022 @07:30PM
(#32684738)
I know, I was just adding context.
Re:Run Stable Diffusion on Your M1 Mac’s GPU
by bm-rf on Thursday September 01, 2022 @02:42PM
(#32681067)
just water cool it! https://www.youtube.com/watch?v=9DyUitTVWlw