Fortunately we have Stability.AI and they release their image generation model already. Hopefully they'll follow with other projects too.

https://stability.ai/blog/stable-diffusion-public-release

There's also a fork that requires a lot less VRAM. I was able to get this working with an Nvidia GTX 1070.

https://github.com/basujindal/stable-diffusion

You'd clone the fork, then download Stability AI's checkpoint, sd-v1-4.ckpt, from https://huggingface.co/CompVis/stable-diffusion-v-1-4-origin...

Follow the instructions in the forked repo, and you should be good to go in a manner of minutes.

The only issue I had was WSL 2 was not using my GPU, so obviously it failed. Couldn't get it working on Windows either, failed at installing dependencies. I'm thinking of dual-booting Linux just to try out stable-diffusion.

I normally dev on Linux but don't do GPU work, only my Windows gaming PC has the horsepower to run stable-diffusion and unfortunately I couldn't get it working on Windows. Shame cause I'd love to play around with it! Does anyone have any tips?

Hm, what issues did you come across on Windows? I have it running on both a Windows PC and a Fedora PC. I uninstalled python, then installed the full installation of Anaconda. Then I followed the "Requirements" section of the readme and I was good to go.

Edit: Just noticed they removed the instructions from the readme for some reason. The requirements section can be found in this diff: https://github.com/basujindal/stable-diffusion/commit/487a0f...

Thanks for the input. I can return with error logs later, but essentially when doing the `conda create env -f environment.yaml` line it would error out partway through. It would download dependencies for a few minutes then fail on one. I tried multiple times and got the same error. I just assumed it was an issue of a dependency not properly supporting windows.

Are you saying you coulda `conda create env -f environment.yaml` fine? This was my first conda install on Windows, so perhaps I missed something else.

Yeah, my Windows machine was able to run that command just fine.

Maybe one of the newer commits broke something? I think I cloned it when b91816301fa62df239a45336b381ee918ff52b2a was the latest, but I'll have to check when I get home.

Yes it works fine for me on Windows with conda.

    conda env create --name envname -f environments.yml

Try reinstalling conda. Do you have a firewall on or anything like that?

Try out a Ubuntu live image. You can probably get set up with everything without even having to touch your HDD! I just don't know how the native NVidia drivers work in that case, but I think you can get them installed and restart the window manager, even w/a a liveCD.

That's a really good idea than finding another HD to install. Thanks.

Same. I've spent a number of nights trying variations on the install instructions. Got the windows CUDA drivers (older graphics card), but no matter what I try, conda, or WSL, pytorch refuses to see the CUDA available.

I'm able to make it works on my GTX 1650 LP. Had to use full precision though or else it generates green images. I also had to reduce the resolution to 320x320 and close every applications that uses vram because the GPU only has 4GB of vram. Took about 2 minutes to generate a single image with ddim sampling steps set to 50.

Does it make images of lesser quality or does it just take longer for the same quality?

Anyone built a Docker image for this? I don't want to duplicate efforts.

Much respect to Stability - they followed through on their promise.

It's probably a lot more than that, $500k is like one Google ML engineer's salary.

Right? I feel like novel development of these models looks more like the GPT 10-20M numbers.

>Every single time I see an article about a new AI model that has a section called "societal impact" I know immediately they are not releasing the model, the training set, nothing...

This section is just a requirement for some of the big ML conferences, like NeurIPS.

And especially so because this looks like a straight up marketing website. This is all about their own product.

It’s not new, it was created shortly after dall-e. To my knowledge, there is no product atm. I’d pay to get access to this api, but you can’t.

I for one am glad that, for once in my life, an obviously huge advancement is taking into account the human impact of releasing the technology responsibly. Maybe it’s just a “BS statement” but given the major strides they’re making at removing racial/gender biases[1] from similar projects, I don’t think it’s just hot air. Especially given the phenomenon of “bias amplification”.

Maybe I’m being too optimistic but either way, given the pace of progress we won’t have to wait very long to play with this magic.

[1] https://openai.com/blog/reducing-bias-and-improving-safety-i...

>the major strides they’re making at removing racial/gender biases

They're literally just appending a race/gender string at the end [1]. In what world is that not just hot air?

[1] https://twitter.com/jd_pressman/status/1549523790060605440

That’s a pretty uncharitable interpretation of my post. You shared an example of a single mitigation that you personally find ludicrous (without explaining why). And then I’m supposed to throw up my hands and go “I guess it’s pointless to try and be less racist”?

Bias amplification is a real issue. https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...

Tay might still be around if Microsoft gave a thought to potential issues before release. I’d prefer not to have this awesome technology tainted out of the gate as a tool for racists and pornographers. They’ll get their hands on it eventually but it’d be nice if they don’t get all the up-front press.

> a single mitigation

That is the only mitigation used in DALL-E 2, which up until recently was the only publicly available text to image model.

> I’d prefer not to have this awesome technology tainted out of the gate as a tool for racists and pornographers

Why is it your business what people do with the model? If people want to be racist they can already do so, they don't need a shitty model that doesn't work half as well as paying some guy in the third world $2/h to shitpost online. And I don't see the problem with pornography.

I just voiced my support for their decision. Just as you are voicing your displeasure with it. I don't see how we are different.

Everything has a “bias” including reality. This type of talk is not about removing biases, it is about imposing biases in line with the wokeligion and attempting to manipulate public view away from the facts of reality. But reality always wins in the end.

Funny how it stopped mattering when OpenAI decided to charge for it.

right, because I can't be trusted to not be racist

Indeed, you can't be trusted not do anything bad with it. Who's going to vet each and every user? Who's going to check every time it's used for the coming 10, 20, 30 years?

If he's racist he can already hire black people to take a photo for an "upcoming action movie" called evil baby. They will be asked to hold guns aiming it at a crib. Then release it on the internet and say he saw 3 black people about to shoot a baby.

It would be called out as fake or staged just like an imagen/walle2/openai would be called out as fake. The thing that makes stories real is real people - actual events and backed up testimonies.

If he wants a picture of a dog with sunglasses in a boat, he can hire a dog, a handler, and a boat. He doesn't need this model for anything.

> The thing that makes stories real

Since when do the hordes care about stories being real? People get harassed over photoshopped images. People get killed over false rumors. These tools make it easier to trigger such reactions.

>hire black people to take a photo for an "upcoming action movie" called evil baby. They will be asked to hold guns aiming it at a crib.

As a side note, I'd watch that. I imagine the next scene shows a bloody room and a baby escaping.

If someone is racist to me online, I'll just turn off my screen

> “While a subset of our training data was filtered to removed noise and undesirable content, such as pornographic imagery and toxic language, we also utilized LAION-400M dataset which is known to contain a wide range of inappropriate content including pornographic imagery, racist slurs, and harmful social stereotypes. Imagen relies on text encoders trained on uncurated web-scale data, and thus inherits the social biases and limitations of large language models. As such, there is a risk that Imagen has encoded harmful stereotypes and representations, which guides our decision to not release Imagen for public use without further safeguards in place.”

Of course that training set was scraped off of everyone else’s web sites, which is fair use. But the model is copyrighted, and I bet you’ll hear an argument that it’s outputs are copyrighted.

The only AI ethics I’m worried about are the lack of anything ethical going on w.r.t intellectual property in AI.

Yes, this is how fair use works. Once copyright agrees that you were in the right to make something, you have exclusive ownership over it, it does not get locked open like GPL software does.

Scraping other people's content is already established fair use thanks to Google prevailing against the Author's Guild in front of SCOTUS. I find it difficult to understand how it could be illegal to scrape a bunch of creative works to create a system that generates new works, but legal to scrape a bunch of creative works to let people search two-page excerpts out of them.

Output copyright is very much up in the air. The Copyright Office has rejected copyright registrations claiming the software itself created the work; but presumably this wouldn't apply to a human taking ownership over something they used ML to create. The amount of prompt engineering you have to do to make these kinds of systems alone would count as some kind of creativity. The only real complaint I could see is if the system regurgitated its training data, which would be bog-standard copyright infringement.

Also, related note: I really hate how the whole AI thing is making the FSF sound like the Author's Guild did a decade and change ago. The law is already very clear that you cannot launder an infringement (copying GPL code) through a fair use (trained ML weights). Please do not adopt the arguments of copyright maximalists.

They will still use it and make a lot of money from it, I guarantee it.

Google is in the business of ads, this might have useful applications in the ad space.

It feels like the wind has shifted because of Stability AI, and now companies doing this are going to look like those that have “call to get a demo” links on their homepage in a SaaS world.

in the case of imagen, I suppose the cost is at least two orders of magnitude over 500k.

... no it definitely wasn't. that's $50m. read the paper, they tell you how long it took on a v4-256, which you know the public rental price for.

Pretty much everybody who trains models trains tens to hundreds of models before their final run. But a better way to think about this is: Google spent money to develop the TPUv4, then paid money to have a bunch built, and they're sitting in data centers, either consuming electricity or not. Google has clearly decided the value of these results exceeds the amortized cost of running the TPU fleet (which is an enormous expense).

While I don't agree with gp's price estimate of $50M, you are also forgetting that to train the final model, they had to iterate on the model for research and development. This imposes a significant multiple over the cost to just train the final model.

they're talking about the total cost of the project, including the salaries of the humans, their health care, their office space, etc.

And where did you think the tagged dataset and software came from?

Marginal cost of using that is basically $0. The internal dataset is a sunk cost that's already been paid for (presumably for their other, revenue generating products like Google Images). Half of their dataset is a publicly available one.

One order of magnitude over 500k it's 5M, two orders is 50M