Boorus [0] contain millions of images, manually labeled to a pretty high quality. Notably defusion models trained on booru datasets have had good success.

This is not the only example of well curated image-tag pairs, especially in artistic circles. It's just that most of them are not CC.

[0]: https://en.wiktionary.org/wiki/booru

Booru use tags instead of captions, so a model trained on them is really limited; moreover, Danbooru has only 5 million images, while other booru such as gelbooru and sankaku have lower quality.

Tags are limited how exactly? Prompt crafting becomes a case of selecting the relevant tags, and the embedding space will still capture the dataset.

Danbooru is only one such example of well curated tagging and if we ignore copyright there are far more examples. These example just serve as evidence that refining poor labeling is not outside of the relm of possibility as you suggested.

A tag-based system would completely lack any kind of contextual information and it would not be possible to create any relationship between words; natural language is much more powerful.

An example, an image is tagged: kanna_kamui, kimono and torhu_(maiddragon), who has the kimono? Kanna, Torhu or both? It cannot be known, but with natural language it is possible to describe who is wearing what.

I think this is mainly theoretical at this point. In my experience, current technology doesn’t seem to be utilizing the additional information that comes from natural language all that well. For example: prompt Dalle2 for “a dinner plate on a stack of pancakes” and you will get ordinary images of pancakes on plates, not the other way around.

Edit: an experiment comparing tags/BOW vs natural language sequences in image generation tasks would be interesting to see.

I think this does not work mainly because of the unusual situation you describe, such as "a horse riding a person"; most of the time Dalle 2 is really good at following the prompt.

>It is not possible to manually label hundreds of millions of images to train a model on them

Citation, please?

I think you mean "the developers of this technology do not want to pay to have hundreds of millions of images labeled".

It is not believable that someone would pay humans to label 400mln or 5bln images/samples to train a model on them, but I guess if you argument is "everything is possible" then gotcha

If it's done in a reCAPTCHA like way, it can be done fairly efficiently and for cheap. In fact Scale AI does just this, they do manual labor operations such as captioning images, as an API. Here's their product for image labeling: https://scale.com/rapid.

Unstable Diffusion is also doing their captioning like how I mentioned, with groups of volunteers as well as hired individuals.

Scale seems to do, for example, image classification but not captioning as it would be hard to compare the results with others people to verify the quality (when you have a discrete number of classes is really straightforward), also can you report where you read about the Unstable Diffusion plan for manually labeling image datasets? I want to dig deeper

From their Reddit post about this: https://old.reddit.com/r/StableDiffusion/comments/zhg18s/uns...

> We are releasing Unstable PhotoReal v0.5 trained on thousands of tirelessly hand-captioned images

They seem to have created a much smaller dataset than LAION's, it would not work to train a generative model on such a small amount of images (obviously the images here do not have a single domain).

You seem to be confusing "possibility" with your personal opinion on what you think would be done by others.

As a human being I know human limitations, explicitly labeling 400mln/5bln images for a particular task seems absurd to me, but if you think it is realistically possible perhaps you can give an example.

The LAION dataset was designed for the broader community at the first place, so clearly the premise is that they don’t have millions to throw at the problem.