How does this work? When I run it on a machine with a GPU (pytorch, CUDA etc installed) I still see it downloading files for each prompt. Is the image being generated on the cloud somewhere or on my local machine? Why the downloads?

Shouldn't be downloads per prompt. Processing happens on your machine. It does download models as needed. A network call per prompt would be a bug.

OK. I noticed that the images are not accurate when I give my own descriptions. Not sure if this is a limitation of Stable Diffusion. For example, for the text "cat and mouse samurai fight in a forest, watched by a porcupine" I got a cat and a mouse (with a cat's face and tail!!) in a forest sort of fighting. But no - porcupine

Thank you for creating this.

yes stable diffusion is not great about handling multiple ideas. New image models coming out soon though.

I keep seeing this even when the prompt is unchanged

Downloading https://huggingface.co/runwayml/stable-diffusion-v1-5/resolv... from huggingface Loading model /home/hrishi/.cache/huggingface/hub/models--runwayml--stable-diffusion-v1-5/snapshots/889b629140e71758e1e0006e355c331a5744b4bf/v1-5-pruned-emaonly.ckpt onto cuda backend...

followed by a download

That is strange. I'm not sure what would cause that unless it was running in some ephemeral environment. What OS? Can you open a github issue with a screenshot?

Ubuntu 22.04.1 LTS. I'll file a issue

Oh I see what you're saying. It's not actually downloading but there is a bug where it's not using the cache properly. will fix

Is there a way to pre-download all models? I want to create a docker image and cache the models.

Here is a colab you can try it in. It crashed for me the first time but worked the second time. https://colab.research.google.com/drive/1rOvQNs0Cmn_yU1bKWjC...

Huh, I'm trying it now and the results seem so weak compared to any other model I've seen since dall-e.

Hmm that’s true for me too. Not sure if it is due to resource constraint. I had a picture of a car in indoor parking lot with walls and pillars. When I prompted “Color the car blue” the whole image was drenched in a tint of blue. Similarly when prompted “make a hummingbird hover” … the hummingbirds were a patch of shiny colors with an shape that sort of looked like an hummingbird but not like a real one.

Try "turn the car blue"

Does dalle do prompt based photo edits now?

But yeah sometimes it doesn't follow directions well. I haven't noticed a pattern yet for why that is.

How would I upgrade to 2.1 if running locally?

If you're wanting to use Stable Diffusion 2.1 with imaginairy you just specify the model with `--model SD-2.1` when running the `aimg imagine` command.

Sorry for the offtopicness but could you please email me at [email protected]? (I have an idea/possibility for you. Nothing that couldn't be posted here in principle but I'd rather not take the thread off topic.)

This was an uncanny comment, somehow.

Hope ya'll brainstorming session is fruitful.

Sorry for the uncanning! My thought was simply to connect the OP with a YC partner in case they wanted to explore doing this as a startup.

I send such emails all the time but on semi-rare occasions have to resort to offtopic pleas like the GP.

I hope that helps clear things up!

It does, thank you!

EDIT - it's free of charge: https://research.google.com/colaboratory/faq.html

---

First time I've used "colab" - looks great. Out of interest, who pays for the compute used by this?

Is it freely offerred by Google? Or is it charged to my Google API account when I use it? Or your account? It wasn't clear in the UI.

Freely offered by Google. They offer a subscription model if you want to run your collab notebooks on a beefier machine.

I could not get the first cell to run.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. tensorflow 2.9.2 requires protobuf<3.20,>=3.9.2, but you have protobuf 3.20.3 which is incompatible. tensorboard 2.9.1 requires protobuf<3.20,>=3.9.2, but you have protobuf 3.20.3 which is incompatible.

I believe you can just ignore that error. The cell ran.

> Here are some examples of transformations it can make: Golden gate bridge:

I'm on mobile so can't try this myself now. Can it add a Klingon bird of prey flying under the Golden Gate Bridge, and will "add a Klingon bird of prey flying under the Golden Gate Bridge" prompt/command be enough?

No. At least not with the Stable Diffusion 1.5 checkpoint used in the colab notebook. It seems to only have a very vague idea of what a Klingon bird of prey is. The best I could get in ~30 images was [1], and that's with slight prompt tweaks and a negative prompt to discourage falcons and eagles.

1: https://i.imgur.com/gDj2Kn4.png

How about “fix the hands”?

The hands issue is going to be an awesome story for all of us in 10-20 years. The younger generation just won’t fathom how hard it was to get proper hands. I wonder what a parallel comparable now would be? Something the slightly technical general public just can’t wrap their head around why it was complicated “back then”.

Maybe todays example is a smart voice assistant like Alexa.

Or maybe it will never be fixed, and in the future when they are trying to determine if someone is a human or an artificial replica, they will simply ask them to draw a set of human hands as a test.

Someone needs to redo the blade runner scene with SD with the hands question :)

Most humans would struggle to draw human hands as well.

Well, this already uses this default negative prompt https://github.com/brycedrennan/imaginAIry/blob/master/imagi... so it may fix the hands automatically.