simandl

Last year's ICLR had a paper, "Data Poisoning Won't Save You From Facial Recognition" that included the Glaze team's previous project, Fawkes. This statement from that paper is quite damning.

"This paper shows that these systems (and, in fact, any poisoning strategy) cannot protect users’ privacy. Worse, we argue that these systems offer a false sense of security. There exists a class of privacy-conscious users who might have otherwise never uploaded their photos to the internet; however who now might do so, under the false belief that data poisoning will protect their privacy. These users are now less private than they were before."

Paper: https://arxiv.org/pdf/2106.14851.pdf Fawkes: https://sandlab.cs.uchicago.edu/fawkes/

This assumes that the filter actually works in practice: https://www.reddit.com/r/StableDiffusion/comments/11v7sv9/ha...

Thank you for the feedback! That's what we're hoping our opt-in tools will help artists do. For the problems you've posed here, you might actually want to both opt-in and opt-out. You'd be able to flag the imageurl-caption pairs that don't accurately credit you or describe your work and we'll forward them to the dataset creators for removal. You could then add any works that you're comfortable being used for AI training, and caption them however you'd like. Those would go into the dataset and be used when training future models.

We'd love to have you sign up if you're interested! We expect to start the beta for those tools in 2 to 3 weeks.

We think this is because the images are all links and the browser itself is pulling them in from across the web. With chrome it happened once during our testing, and we've had one user also experience it, but it's intermittent. Thank you for pointing to brave, which we haven't tested yet! That might help us repeat the errors.

These are important questions. We aren't storing any of the images used to search, and the email addresses are going to mailchimp lists for opt in and opt out. As we role out the next set of tools, which let users flag images and create lists from them, we'll email the mailchimp lists with more info.

When we enable sign-in, we'll also add a privacy policy, because at that point, we will store some images, on request, to use them to make finding other works by the same artists easier.

Opt-out image URL lists will be made available to the dataset owners for removal. Opt-in image lists will be public.

They have their own datasets and included Laion-400M, a subset of 5b that was released prior to 5b. You can see a short explanation in imagen's "Limitations and Societal Impact" section at: https://imagen.research.google/.

> While a subset of our training data was filtered to removed noise and undesirable content, such as pornographic imagery and toxic language, we also utilized LAION-400M dataset which is known to contain a wide range of inappropriate content including pornographic imagery, racist slurs, and harmful social stereotypes.

We (Spawning) did not create the dataset or train the models in question. We're working to make it easy for people to remove themselves from, or add themselves to, this dataset and future models.

This article highlights several of the problems that we are working on. Here's another article, about our organization:

https://www.inputmag.com/culture/mat-dryhurst-holly-herndon-...

Reading through your comments, it appears you're under the impression that we (Spawning) trained models using this data. That is not the case. We're building tools to help people to remove themselves from, or add themselves to, datasets used (by others) to train these models. We're hoping to make that as close to 0 effort as possible for all parties. We agree that artists should control how their works are used, and we're working hard to make that the norm.

Our initial approach will be to validate the artists manually, and trust them to only flag or upload their own works. If we hit a scale where that starts to become an issue, we have plenty of ideas for mitigating this, but we're not optimizing for that problem yet.

You might be surprised. We have almost as many opt-in requests as opt-outs since we announced this today.

We don't see this as binary in the long term. Maybe artists want to release art from a prior period, for example, but withhold their current series until they move to the next.

We don't store any images used for searching.

We are building an opt-in list, because a lot of people do want to be able to prompt AI with something like, "a cat in the style of me" or "me riding a dinosaur". That will be shared publicly, of course.

Thanks! That is definitely on the list, but might be a few months away. We're focusing on using images to find other images so it will be easy for artists to flag all of their stuff quickly. But, once we have that in a good place, we will definitely be adding more to the text search side.

If that's a rights issue, we'll definitely add a link to the source. For now, you can right click -> open in new tab to see where it came from, but we'll look into this asap.

The goal here is to give people the opportunity to remove images they don't want in this dataset or add images they do want in there.

For sure! And as artists opt in, you'll be able to use it to see how they describe their work.

Please do sign up and you'll be able to flag these images soon. We'll work to get them removed from this and future datasets built for AI training.

It's using openai's clip (https://openai.com/blog/clip/) to find the image similar to your query or image. Clip learned to match images to the captions that were paired with them from images on the web. My best guess would be that there were enough pictures from someone with your name (maybe just last name) that it made that association.

Yes, that's the same dataset. This website has some additional tools coming so artists can flag and opt out, or upload to opt in, and we'll get those to the laion team to add or remove from the 5B (and future) datasets.

I'm not 100% on this, but I think a big portion of the captions for the images come from the alt-text, and are probably auto-generated by the sites they were scraped from.

This is Laion-5B, https://laion.ai/blog/laion-5b/

It's built off of common crawl, so it probably does have a pretty representative sample from whatever the big image searches use.

Funny enough, the NSFW filter that laion built is turned on. Without it, it's... a lot. The NSFW stuff is done with a model, so you get a probability of NSFW out of it, and you can select a threshold.

If you set the threshold high, like 95% certainty that an image is nsfw to filter it, you get a bunch of false negatives, letting a ton of nsfw through. Set it too low, and you throw out stuff that isn't nsfw.

We (haveibeentrained) erred on the side of too high, so we wouldn't tell artists their work wasn't in there if it was. Tough trade-off there. Similar to using the dataset to train an AI model, where you might cut off useful images from training if you try and filter all the nsfw.

This is clip matching the text or image searched to the images in the Laion-5B dataset.

It's using clip to match the text to the image, so you can actually prompt it like you might an art generator. Here's "in the style of greg rutkowski": https://haveibeentrained.com/?search_text=in%20the%20style%2...

In the next few weeks we'll be adding the ability to log in and flag or upload your works (if they aren't there). Those lists will have permissions assigned to them, starting with simple opt-in or opt-out.

Stable Diffusion used an aesthetic filter to train on a subset of the English language images from this full 5.8 billion multi-language set. That probably got a lot of what you're finding.

This is Laion-5B, you can read more about it here: https://laion.ai/blog/laion-5b/

Imagen and Stable-Diffusion both used subsets of this full 5.8B image set.

This is a shiny + d3 side project I'm working on. It's still an MVP and I'd appreciate any feedback on improvements.