[0] https://huggingface.co/CompVis/stable-diffusion [1] https://github.com/CompVis/stable-diffusion

Good luck!

From the GitHub's README:

    sd-v1-1.ckpt: 237k steps at resolution 256x256 on laion2B-en. 194k steps at resolution 512x512 on laion-high-resolution (170M examples from LAION-5B with resolution >= 1024x1024).

    sd-v1-2.ckpt: Resumed from sd-v1-1.ckpt. 515k steps at resolution 512x512 on laion-aesthetics v2 5+ (a subset of laion2B-en with estimated aesthetics score > 5.0, and additionally filtered to images with an original size >= 512x512, and an estimated watermark probability < 0.5. The watermark estimate is from the LAION-5B metadata, the aesthetics score is estimated using the LAION-Aesthetics Predictor V2).

    sd-v1-3.ckpt: Resumed from sd-v1-2.ckpt. 195k steps at resolution 512x512 on "laion-aesthetics v2 5+" and 10% dropping of the text-conditioning to improve classifier-free guidance sampling.

    sd-v1-4.ckpt: Resumed from sd-v1-2.ckpt. 225k steps at resolution 512x512 on "laion-aesthetics v2 5+" and 10% dropping of the text-conditioning to improve classifier-free guidance sampling.

-----

Sorry my bad, found the answer. One simply adds the following flags to the StableDiffusionPipeline.from_pretrained call in the example: revision="fp16", torch_dtype=torch.float16

Found it in this blogpost: https://huggingface.co/blog/stable_diffusion

mempko thank you for your hint! I was about to drop a not insignificant amount of money on a new GPU.

What does one lose by using float16 representation? Does it make the images visually less detailed? Or how can one reason about this?

Edit: Just to be clear, your intuition that it could cause issues is certainly merited - and not _all_ models can be trivially converted from fp32 to fp16 without some new error accumulating (during inference). Variational autoencoders like VQGAN and GAN's are particularly prone to such issues.

But in this case, it's all upside.

One time:

1. Have "conda" installed.

2. clone https://github.com/CompVis/stable-diffusion

3. `conda env create -f environment.yaml`

4. activate the Venv with `conda activate ldm`

5. Download weights from https://huggingface.co/CompVis/stable-diffusion-v-1-4-origin... (requires registration).

6. `mkdir -p models/ldm/stable-diffusion-v1/`

7. `ln -s <path/to/model.ckpt> models/ldm/stable-diffusion-v1/model.ckpt`. (you can download the other version of the model, like v1-1, v1-2, and v1-3 and symlink them instead if you prefer).

To run:

1. activate venv with `conda activate ldm` (unless still in a prompt running inside the venv).

2. `python scripts/txt2img.py --prompt "a photograph of an astronaut riding a horse" --plms`.

Also there is a safety filter in the code that will black out NSFW or otherwise expected to be offensive images (presumably also including things like swastikas, gore, etc). It is trivial to disable by editing the source if you want.

Unfortunately I'm getting this error message (Win11, 3080 10GB):

> RuntimeError: CUDA out of memory. Tried to allocate 3.00 GiB (GPU 0; 10.00 GiB total capacity; 5.62 GiB already allocated; 1.80 GiB free; 5.74 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CON

Edit:

>>> from GPUtil import showUtilization as gpu_usage

>>> gpu_usage()

| ID | GPU | MEM |

------------------

| 0 | 1% | 6% |

Edit 2:

Got this optimized fork to work: https://github.com/basujindal/stable-diffusion

You should check their threads there, there's some good info.

    cog predict r8.im/stability-ai/stable-diffusion -i prompt="cow flying in space"

I built a simple UI around this, which installs Stable Diffusion's docker image, and lets you play with it locally in a browser-based UI. https://github.com/cmdr2/stable-diffusion-ui

> And log in on your machine using the huggingface-cli login command.

I find that annoying. I guess it is what it is.

Here's the direct link.

https://old.reddit.com/r/StableDiffusion/comments/wuyu2u/how...