They probably still going to adapt the work to fit their needs, and so companies probably still want actual artists for entertainment and artistic purposes.
Small companies (and individuals) probably can use it to circumvent costly stock images all together, so the lower tier photographers/artists there gonna have a problem.
[1] https://colab.research.google.com/github/TheLastBen/fast-sta...
The main challenge is finding the right balance between "make something that looks exactly like this" and "put it in a completely different context". Better similarity equals less flexibility.
For now, the most effective combination will be artist + AI, although it does feel a bit like those that incorporate it in their workflow are helping to dig their own grave.
1 Generate a bunch of characters or objects based on a prompt.
2 Pick the one you like
3 Tell the AI to extract the character's traits from that one picture
4 Miracle happens (I don't know. What does a "character definition file" look like?)
5 Make a new prompt, but add the character definition from step 4, so you get the same traits, only in a different setting or position, etc
Note that training is extremely expensive, and is beyond the capabilities of most end users. Here are the details of their training method:
> Given ~3-5 images of a subject we fine tune a text-to-image diffusion in two steps: (a) fine tuning the low-resolution text-to-image model with the input images paired with a text prompt containing a unique identifier and the name of the class the subject belongs to (e.g., "A photo of a [T] dog”), in parallel, we apply a class-specific prior preservation loss, which leverages the semantic prior that the model has on the class and encourages it to generate diverse instances belong to the subject's class by injecting the class name in the text prompt (e.g., "A photo of a dog”). (b) fine-tuning the super resolution components with pairs of low-resolution and high-resolution images taken from our input images set, which enables us to maintain high-fidelity to small details of the subject.
Each fine-tuned model is a copy of the original model. So if the model is 10GB, the fine tuned version will be a separate 10GB file. That might not sound like a lot, but it quickly adds up.
In this case, end users are artists. One could imagine a cloud-based art program which will fine tune on demand. That certainly seems like a good startup idea.
You can't run it on consumer hardware, but you can just rent a GPU (or use a free collab book) for a few hours to generate the model. Then you download and reuse it locally at will. Yes, you need storage and if you train often it can get expensive, but it is by no mean out of the end user, at least professional end users, capabilities. And of course there are growing libraries of freely available pretrained models.
> In this case, end users are artists. One could imagine a cloud-based art program which will fine tune on demand. That certainly seems like a good startup idea.
Very much agree about this. At least for a while, I strongly believe that AI will just be another tool for artists willing to embrace it, far from replacing them.
This area is moving really fast, one-click solutions are already being created.
People are also averaging weights of multiple models to create new models based on multiple other dreambooth models, and it works surprisingly well.
The stable diffusion reddit is a good place to see how all of this is developing.
I have an rtx 2070 with 8gb and it has been working quite well for me. However there are always models that will not fit. For those running on the cpu with potential nvme offload is not that bad. For example a single inference on bloom 7b (30gb of ram required just for weights) on a 32gb ram machine takes about 30s (it has to offload few GB to nvme). This is on zen 3 ryzen and with no gpu use. I can't wait to try cpus that support avx512.
Do you happen to have a link to that YouTube video you followed?
You can use it to make procedurally-generated art, but it's still obvious what the source material was.
DreamBooth really should have destroyed all doubt about this point point, as the AI can generated highly specific images of subjects that aren't even in the original training set.
I haven't seen any concrete examples of this yet.
This is a common misunderstanding. Image AIs don’t copy/paste anything, or read from a database. Stable Diffusion is only 2GB. It’s little more than a huge array of numbers, which relate to different tokens.
Finding the “midpoint” between a set of tokens is latent space. What emerges is nothing short of miraculous, but it comes from random noise, not copy/pasting anything.
One obvious counterexample is stylegan interpolations. If you interpolate between two images, the midpoint is usually unique — it’s often not obvious what the source material was. (E.g. Gwern’s anime interpolations; sure, they’re anime faces, but from where? “All of danbooru” might as well be “all styles of anime ever created.”)
Maybe someone else can argue the point further.
There's a lot of good discussion to be had around the ethics of AI art, training on copyrighted materials, etc. But it is equivocally not just copy pasting from a database of images.
> But it is equivocally not just copy pasting from a database of images.
Just because you put a lossy compression step in the middle doesn't mean it's not copy-paste anymore.
With this tool suddenly the effort of making a series of compositions that are coherent will dramatically change the game, specially in the videogame industry. I think there are a lot of programmers out there capable of making great games but might be lacking the resources to fully complete their visions due to having to needing assets for their games.
It seems to me like the approach stable diffusion has taken has dramatically increased interest and utility of these tools. So I'm hoping they follow similar lines for other types of AI. Every week I'm reading of a new novel use for these generators that I hadn't really considered before.