Re:High-performance image generation using Stable Diffusion ...
by CrazyStat on Wednesday September 28, 2022 @11:01AM
(#33008782)
This is the compositionality problem--the language model sometimes doesn't quite know how to put the words together. Better language models will help in the future; in the mean time you can give it a helping hand by prompt engineering or using img2img.
Re:High-performance image generation using Stable Diffusion ...
by CuriouslyC on Wednesday September 28, 2022 @09:05AM
(#33007414)
Graphic designers lean on img2img in their workflows more than txt2img, as that gives you the control you speak of.
Re:High-performance image generation using Stable Diffusion ...
by rytill on Wednesday September 28, 2022 @07:38PM
(#33014592)
Do you know of examples of graphic designers who have shared their img2img workflows online?
Re:High-performance image generation using Stable Diffusion ...
by CuriouslyC on Wednesday September 28, 2022 @08:43PM
(#33015028)
Hang out in reddit.com/r/stablediffusion, people have posted workflows quite a bit.
Re:High-performance image generation using Stable Diffusion ...
by UncleEntity on Wednesday September 28, 2022 @01:40PM
(#33010782)
My favorite is when you do “<whatever> bla, bla, bla, wearing a t-shirt by <artist>“ and it gives you an image of <whatever> wearing a t-shirt with a print in the style of the artist. Which adds extra dimensions to play with so isn’t all that bad.
Re:High-performance image generation using Stable Diffusion ...
by nanonomad on Thursday September 29, 2022 @04:08PM
(#33026401)
https://energy-based-model.github.io/Compositional-Visual-Ge...
"
Compositional generation. Our method can compose multiple diffusion models during inference and generate images containing all the concepts described in the inputs without further training"
Re:High-performance image generation using Stable Diffusion ...
by overengineer on Thursday September 29, 2022 @01:13PM
(#33023894)
There must be a way to disambiguity of the prompt by adding some negative factor to clip guidance penalizing possible misunderstandings of the prompt maybe
The prompt was "A cute otter in a rainbow whirlpool holding shells, watercolor"
Seems like the otter should be holding shells, the way a normal human parses it.
The tool showed the otter 'in holding-shells', which are shells that hold otters apparently. Also some random shells strewn about, as the technique is sensitive to spurious detail sprouting up from single words.
Until the tool permits some kind of syntactic diagramming or so forth, we'll not be able to control for this.
Just the other day here, I saw a picture of a fork and some plastic mushrooms. The prompt was 'plastic eating mushrooms' which was ambiguous even to humans. The tool chose to illustrate the subclass of mushrooms 'eating-mushrooms' (as opposed to poison mushrooms or decorative mushrooms I suppose) made of plastic.
When we're playing around this can seem whimsical and artistic. But a graphic designer might want some semblance of control over the process.
Not sure how a solution would work.