Running Stable Diffusion XL 1.0 in 298MB of RAM

Re:Running Stable Diffusion XL 1.0 in 298MB of RAM by brucethemoose2 on Tuesday October 03, 2023 @04:42PM (#37757263)

This is the best I've tried so far, but no mac support I don't think. Its a feature packed fork of Fooocus, which was developed by the orginal ControlNet dev. The quality you can get from small prompts is mind boggling:

https://github.com/MoonRide303/Fooocus-MRE

For base SD 1.5, I use Volta, because its fast: https://github.com/VoltaML/voltaML-fast-stable-diffusion/com...

Really good SD 1.5 image quality comes from gratuitous use of finetunes, LORAs, controlnet and other augmentations. So you can, say, trace a base image for structure, specify prompting in certain areas of the image and so on. InvokeAI is actually quite feature packed, and has lots of these augmentations hidden in the nodes UI, but Volta and other UIs also expose them more directly.

[ Parent | on HN ]

Re:Running Stable Diffusion XL 1.0 in 298MB of RAM by Cannabat on Wednesday October 04, 2023 @07:48AM (#37763801)

Fooocus does quite a bit of prompt massaging for you - there are models that take a few words and turn them into “prompt engineer” level prompts. Makes a huge difference.

[ Parent | on HN ]

Re:Running Stable Diffusion XL 1.0 in 298MB of RAM by brucethemoose2 on Wednesday October 04, 2023 @12:01PM (#37767124)

Yeah, and InvokeAI has a similar "IP-adapter" model.

Still, even with it turned off, the quality is quite remarkable.

[ Parent | on HN ]

Re:Running Stable Diffusion XL 1.0 in 298MB of RAM by Cannabat on Wednesday October 04, 2023 @03:45PM (#37770583)

Ip adapter is a bit different from what fooocus and midjourney do.

Ip adapter uses an image to guide denoising.

Fooocus and MJ take a prompt and expand it in a variety of ways (eg a language model or more simplistic text manipulation). The actual prompt that creates the conditioning is not what you typed in. That’s what I mean by prompt massaging

[ Parent | on HN ]