News for hackers. Stuff that matters.

A comment on Stable Diffusion XL technical report [pdf]. Parent comment

Re:Stable Diffusion XL technical report [pdf] by rhogar on Tuesday July 04, 2023 @11:02AM (#36587606)
The report does not detail hardware -- though it states that SDXL has 2.6B parameters in its UNet component, compared to SD 1.4/1.5 with 860M and SD 2.0/2.1 with 865M. So SDXL has roughly 3x more UNet parameters. In January, MosaicML claimed a model comparable to Stable Diffusion V2 could be trained with 79,000 A100-hours in 13 days. Some sort of inference can be made from this information, would be interested to hear someone with more insight here provide more perspective.
[ Parent | on HN ]
Re:Stable Diffusion XL technical report [pdf] by darqwolff on Tuesday July 04, 2023 @12:23PM (#36588821)
wouldnt that mean more vram is required to load the model? they are claiming it will still work on 8 gb cards.
[ Parent | on HN ]
Re:Stable Diffusion XL technical report [pdf] by Filligree on Tuesday July 04, 2023 @12:29PM (#36588931)
Stable Diffusion 1/2 were made to run on cards with as little as 3GB of memory.

Using the same techniques, yes, this will fit in 8.

[ Parent | on HN ]
Re:Stable Diffusion XL technical report [pdf] by brucethemoose2 on Tuesday July 04, 2023 @05:26PM (#36592700)
I am guessing 8 bit quantization will be a thing for SDXL.

It should be easy(TM) with bitsandbytes, or ML compiler frameworks.

[ Parent | on HN ]
Re:Stable Diffusion XL technical report [pdf] by GaggiX on Tuesday July 04, 2023 @07:22PM (#36593756)
bitsandbytes is only used during training with these models tho (the 8-bit Adamw) quantizing the weights and the activations to a range of 256 values when the model needs to output a range 256 values creates noticeable artifacts as they are not going to map 1-to-1.
[ Parent | on HN ]
Re:Stable Diffusion XL technical report [pdf] by liuliu on Tuesday July 04, 2023 @09:19PM (#36594511)
Draw Things recently released a 8-bit quantized SD model that has comparable output as the FP16. It does use k-means based LUT and separate weights into blocks to minimize quantization errors.
[ Parent | on HN ]
Re:Stable Diffusion XL technical report [pdf] by GaggiX on Wednesday July 05, 2023 @07:13AM (#36598680)
I was going to search on the internet about it, but then I realized you are the author (and there is nothing online I think). I imagine that the activations are left in FP16 and the weights are converted in FP16 during inference, right?

Btw very cool

[ Parent | on HN ]
Re:Stable Diffusion XL technical report [pdf] by liuliu on Wednesday July 05, 2023 @12:22PM (#36602655)
Yes, computes are carried out in FP16 (so there is no compute efficiency gains, might be latency reductions due to memory-bandwidth saving). These savings are not realized yet because no custom kernels introduced yet.
[ Parent | on HN ]