News for hackers. Stuff that matters.
Re:Running Stable Diffusion XL 1.0 in 298MB of RAM by julienchastang on Tuesday October 03, 2023 @04:03PM (#37756814)
I've been using Stable Diffusion on a MBP via invoke.ai. Are there recommendations for better parameterization of SD? I can never match the quality of the images I find on the internet even when using the same prompt and (seemingly) the same knobs (e.g., same Model like Euler A, etc). [edited for clarification]
Re:Running Stable Diffusion XL 1.0 in 298MB of RAM by brucethemoose2 on Tuesday October 03, 2023 @04:42PM (#37757263)
This is the best I've tried so far, but no mac support I don't think. Its a feature packed fork of Fooocus, which was developed by the orginal ControlNet dev. The quality you can get from small prompts is mind boggling:

https://github.com/MoonRide303/Fooocus-MRE

For base SD 1.5, I use Volta, because its fast: https://github.com/VoltaML/voltaML-fast-stable-diffusion/com...

Really good SD 1.5 image quality comes from gratuitous use of finetunes, LORAs, controlnet and other augmentations. So you can, say, trace a base image for structure, specify prompting in certain areas of the image and so on. InvokeAI is actually quite feature packed, and has lots of these augmentations hidden in the nodes UI, but Volta and other UIs also expose them more directly.

Re:Running Stable Diffusion XL 1.0 in 298MB of RAM by Cannabat on Wednesday October 04, 2023 @07:48AM (#37763801)
Fooocus does quite a bit of prompt massaging for you - there are models that take a few words and turn them into “prompt engineer” level prompts. Makes a huge difference.
Re:Running Stable Diffusion XL 1.0 in 298MB of RAM by brucethemoose2 on Wednesday October 04, 2023 @12:01PM (#37767124)
Yeah, and InvokeAI has a similar "IP-adapter" model.

Still, even with it turned off, the quality is quite remarkable.

Re:Running Stable Diffusion XL 1.0 in 298MB of RAM by Cannabat on Wednesday October 04, 2023 @03:45PM (#37770583)
Ip adapter is a bit different from what fooocus and midjourney do.

Ip adapter uses an image to guide denoising.

Fooocus and MJ take a prompt and expand it in a variety of ways (eg a language model or more simplistic text manipulation). The actual prompt that creates the conditioning is not what you typed in. That’s what I mean by prompt massaging

Re:Running Stable Diffusion XL 1.0 in 298MB of RAM by sbierwagen on Tuesday October 03, 2023 @04:19PM (#37757000)
Do they specify it's straight from the generator? The process videos I've seen start with "a girl standing in a green field" and then an hour plus of inpainting to fix hands, pose, etc.
Re:Running Stable Diffusion XL 1.0 in 298MB of RAM by doublebind on Tuesday October 03, 2023 @04:46PM (#37757327)
I have the same experience with Invoke.ai or MochiDiffusion in the MBP M1. I can only match the quality of other images with Automatic1111 (https://github.com/AUTOMATIC1111/stable-diffusion-webui).

You’ll need more time and memory compared to Invoke or an Nvidia graphics card, but it’s not that bad: 1-2 s/it for an image in standard 512x768px quality, 14-20 s/it for an image in high 1024x1536px quality (Hires Fix).

Re:Running Stable Diffusion XL 1.0 in 298MB of RAM by IKantRead on Tuesday October 03, 2023 @04:48PM (#37757356)
Are you using custom weights? I'm assuming you are but there is a major difference between using the default RunwayML 1.5 weights and using a model finetuned for a specific purpose.

Generally the trade off is that any of the impressive finetuned models are far less generalizable then the default weights, but in practice this is not a big deal and the results can be a substantial improvement.

Re:Running Stable Diffusion XL 1.0 in 298MB of RAM by liuliu on Tuesday October 03, 2023 @08:42PM (#37759532)
Draw Things added CUDA compatibility seed mode allows you to match NVDIA card generated images on Mac.