Top 1% is a bit exaggerated, but there is definitely a lot of not good stuff. I find that Dall-E does especially poorly with underspecified prompts too, unlike something like Midjourney which can give visually pleasing photos for even the most abstract concepts. Dall-E tends to do better with concrete and specific prompts.

Here's an example: Stressful Shapes

Dall-E: https://i.imgur.com/JBkSh0y.png

Midjourney: https://i.imgur.com/C02Zq3i.png

On the other hand, here's a specific prompt: "nerdy yellow duck reading a magical book full of spells"

Dall-E: https://i.imgur.com/FMKZ8zc.png

Midjourney: https://i.imgur.com/lpsg6af.png

But still "king of belgium giving a speech to an audience, but the audience members are cucumbers" is very specific.

And I don't see the king of Belgium anywhere, two pictures have absolutely nothing to do with the prompt (no king, no speech, no audience, no cucumber), one has the speech and audience but no king or cucumber. Graphically, they are deep into the uncanny valley. Only the third image is kind of right, if you really stretch your imagination.

I’ve been comparing Dall-E, MidJourney, and StableDiffusion. Goes to show how much training set and implementation choices matter. But in all cases, you have to think of the underlying labeled text-to-image sets as paint colors to mix, and prepare a palette accordingly. Still haven’t figured out how to get what I want, but to your point, one can get closer.

- - -

Not sure if this is why, but with OpenAI’s Dall-E, you can’t use public figures. You can use proxies, such as “60 year old banker with salt and pepper hair” and then fill in the rest, e.g. “handsome 60 year old banker with salt and pepper hair giving a speech while standing above 12 cucumbers”:

https://i.imgur.com/qYKOWM1.jpg

Telling it oil painting can fudge who the person is, then pick one that’s close and generate variations:

https://i.imgur.com/QRbV7aM.jpg

Or use a reasonable photo and then use edit and in-painting to try to improve the implausible subject. This takes a photo from the first prompt above, erases the lower half of image, and makes a new prompt for the lower half, while keeping just enough of the upper half to orient the collage, e.g. “[photo_edit] + banker giving a speech to cucumbers bin full of cucumbers”:

https://i.imgur.com/OmOK1HF.jpg

- - -

Over on MidJourney, where it’s happy to use public figures so long as you’re not violating terms of service about their use, first a couple prompt experiments with King Philippe of the Belgians.

https://i.imgur.com/KkgIz2w.jpg

https://i.imgur.com/ekf9ypG.jpg

Then one upsized plausible painting from among those, where the actual command was “King Philippe of Belgium talking in a large group of cucumbers --q 2 --uplight” which is pretty basic.

https://i.imgur.com/QWUaNFv.jpg

> you have to think of the underlying labeled text-to-image sets as paint colors to mix, and prepare a palette accordingly.

Very insightful tip on how to harness the "creativity" of Dall-E and the like.

I see how the phrase "king of belgium" was too vague for Dall-E, so it didn't produce anything recognizable - but changing the words into known details, like "banker" and "salt and pepper hair", worked effectively to generate concrete imagery.

Hilarious results. :)

It's not that it's "vague", they intentionally throw off when you try to generate a photo of a named person. It's an intentional protection they put in. If you just do "king" it'll likely do fine, but if it's referring to a specific person it won't.

Ah I see what you mean - "king of belgium" is a real person, so they put in some safe guards in DALL-E to prevent recognizable images for such queries. Makes sense.

IIRC, DALL-E filters requests related to politicians/celebrities. A friend had tried to make some funny stuff involving the Greek PM a couple of months ago, and it plainly refused. Now, it seems to process the request, but it will not show anyone resembling the person you asked for.

Could that have to do with Sophie Wilmès, Belgian PM 2019–2020? (https://en.wikipedia.org/wiki/Prime_Minister_of_Belgium#Livi...)

I gather Midjourney was trained primarily using Journey album covers?

I find Midjourney to be biased towards an artistic representation (for some definition of artistic) When Dall-e is happy to produce children's scribbles or poor imitations.

Try 'poorly drawn ... by a 5 year old using crayons' in Midjourney.

Even then Midjourney is more high-quality :)

See https://imgur.com/gallery/U5zJMcU

Comparison of two prompts, "poorly futuristic landscape by a 5 year-old" and "poorly drawnn highly detailed futuristic landscape dotted by mahcinery and tall buildings by a 5 year-old"

Also, https://imgur.com/gallery/jvEClos

Comparison of "poorly drawn red sports car in the street of a city by a 5 year-old"

Edit: forgot about crayons :D

That's honestly one of my favorite prompts. It's funny to think I use this state of the art AI to generate crayon drawings, but they look so great!

https://i.imgur.com/jKcNkat.png

Now that I'm aware and biased, DALL-E's first image indeed looks very much like stock photo training. This would also make sense given how they can correlate the image with words completely for free due to pretty extensive metadata.

What puzzles me is if the Getty Images logo can sometimes appear. If you only have a Getty account, you get rid of the logo and can legally use them royalty free?

No, but you can input your getty image to StableDiffussion img2img and see what's out

> On the other hand, here's a specific prompt: "nerdy yellow duck reading a magical book full of spells"

> Dall-E: https://i.imgur.com/FMKZ8zc.png

How well it learned all the common prejudices!

"nerdy" == wears glasses

I'm applauding.

I'm looking already forward to AGI based on the current approaches… It will lead us finally into a better world, for sure. /s

How do you visually show 'nerdy' without resorting to the glasses stereotype? Your prompt is specifically requesting a prejudiced image.

Sure. And the AI serves the expected stereotype.

Isn't that great? The world will become a better place with AI everywhere.

We need especially more AI in law enforcement, and such…

AI should make important decisions. Because it bears the same prejudices as humans. So it can replace humans just great. ;-)

The AI is making no decision. The person entering the prompt made the decision to include the term. It is performing the same function as a pencil.

Now, if 'criminal' rendered as a black male 90% of the time rather than a crouched white male wearing a cheesy burglar mask and a sack over his shoulder, then I could see your point about perpetuating prejudice rather than stereotypes.

Op constructed a horrible prompt. First of all, using king Philippe I. is against the ToS, so let's go with a generic "king".

Let's not confuse the AI with "buts", just say that he is giving the speech to cucumbers.

Lastly, specify some style, because this would probably not work out as a photo.

My single try is not bad at all and it could definitely be improved.

https://labs.openai.com/s/3OUmUxKefJCeLhAk4hkeKX4V

I tried it with Stable Diffusion as well. You can use actual people and the model is even pretty decent at many of the famous ones.

On the other hand, it is more difficult to get it to produce absurd results like these.

my prompt: King Philippe I. of Belgium giving a speech surrounded by [[[[large green vertical cucumbers]]]], digital art in the style of Greg Rutkowski

https://files.catbox.moe/1ej1a4.png

Is the usage of the surrounding brackets some kind of keyword weights specific to stable diffusion?

I think it's specific to SD. [Square brackets] increase the weight while (simple brackets) decrease the weight. In the cucumber case I used them to force the model to take into account the less believable part of the prompt, because otherwise stable diffusion often ignores such parts.

There is always some cherry-picking, but prompt engineering is an art per se, you become better and better working on it. I just started this experiment https://www.instagram.com/unshushproject or without Instagram https://unshush.com and spent A LOT of hours and patience to become good at it. Now I'm very proud of my results and I'm working on doing better.

It's a bit risky to invest too much time because every generator is different and they change the underlying model frequently (see the beta of MidJourney yesterday), but if you do it for passion or curiosity there is no problem.

Now I'm experimenting with a local installation of Stable Diffusion (well, not really "local" because I have an old computer) and the prompt is only one of the things you can tweak. There are num_inference_steps, guidance_scale and other parameters.

Prompt Engineering can help a lot but yes, you're basically right: People are generating many, many images and sharing only the best ones with the fewest artifacts.

For simple prompts with little additional guidance, all the diffusion image generators I've seen/used will produce output about like what the author linked most of the time. There are always a few gems, and honing in via prompt engineering helps immensely.

I disagree, with proper promptcrafting you can expect far far better results than the one in the op, before any cherry picking. (see my comment sibling to yours)

With Dalle-2 I get a satisfying result in >50% of attempts and I'm a beginner.

With Midjourney the result almost always looks great, but often misses some part of what I wanted. I'd say Stable Diffusion is similar. The results are seldom crap, but it's difficult to bend it to produce unusual situations. And in SD it's difficult to keep the entire objects in the frame, but that's a different problem.

Um, have you read the prompt? It looking weird is simply the result of "the audience members are cucumbers". The more crazy your prompt is, the worse the results will generally get.

On top of that DALL-E2 has generally issues with anything dealing with multiple objects. A single person will render fine, groups of people will generally give artifacts. Attributes will also be spread across all objects in the scenes, not just the ones you specified in your prompt, so doing anything more complex will require manual uncropping und inpainting, not just a single prompt.

Anyway, if you avoid the obvious weak spots and holes in the training set, DALL-E2 output is for most part pretty amazing out of the box. It's really more a top 50% than a top 1%.

The biggest bias when it comes to published DALL-E2 images are the prompts. Most prompts you see online are not the actual prompts, but funny descriptions made by a human after the fact. The actual prompt are often much longer and sometimes completely different.

I have found being as direct as possible and removing duplicate or superfluous words works best.

Perhaps this rewrite may yield better results:

"King of Belgium gives a speech to an audience of cucumbers"

I've been reading some folks saying that "prompt engineering" is a legit future vocation in a world where AI has taken over a lot of creative work

And from my experience getting high-quality output from AIs takes a bit of finesse. Not quite unlike crafting a good Google query

so... yes

doubtful - I'm tuning GPT3 with good midjourney prompts as we speak

Is command language the ultimate interface? I doubt it. Similar to how GUI supersedes CLI in most use cases, we should be able to indicate "warmer" / "colder" preference to generate new images from previous attempts.

I'm not a huge fan of controlling things with text prompts but it does seem to be the best way to describe the image you're looking for

For diversity, Dalle 2 has a random chance of injecting "women" or "black" after a prompt. When this happens, at least for me, it generally destroyed the quality of the images. Probably "King" was identified as a gendered word. You can find some discussion of this on the subreddit r/dalle2. Sometimes the images are quite poor, but in this case, openAI is doing additional tampering.

A twitter user figured out which words they were using by generating a lot of images with the starting prompt "A sign being held that says "

As I mentioned here a couple of weeks ago [1], I tested DALL-E with prompts for paintings and drawings in three standard genres: still life, landscape, and portrait. The prompts for portraits yielded a lot of grotesquely unacceptable faces, but almost all of the DALL-E output for the still lifes and landscapes was perfectly fine.

[1] https://news.ycombinator.com/item?id=32433821

They are the worst I’ve seen as well.

Yes, people tend to share the best of the best. However these results seem especially bad, like bottom 10% bad.

Yes, "prompt engineering" in actually a thing. People shares various tips and tricks on the internet to engineer their prompts for best results. Lots of trial and errors required.

Example: https://news.ycombinator.com/item?id=32088718

Of course people are more likely to share the best iamges – or in this case, the one most illustrative of their concern (about watermarks).

Also: my sense is that getting the best results often requires a lot of extra coaching with style/detail words. As we can't see the prompt here, we don't know what sort of style/details were requested. GIGO.

You're right. This shows the prompt and it doesn't have such style directives

https://ibb.co/gz5RDkB

Also, a construction like 'but' that tries to override another expectation may be suboptimal. I gave the same concept a few tries, with more 'sweeteners'. First batch, for prompt "news photo of the King of Belgium giving a speech to an audience that is entirely cucumbers, award-winning, well-composed, detailed surroundings" – & it's a bit better:

https://labs.openai.com/s/9YF5WxF1GoZVdzpLBAQYp2Zg

https://labs.openai.com/s/wBfHevs9hIZXvzkJ686mFmn3

https://labs.openai.com/s/M0i029fZnYjQXHFobUpw7eun (best of batch imo)

https://labs.openai.com/s/Hf4z0M9M3KBr6IaaKsjEt9Mx

A few more tries didn't manage to create any photorealistic shots with actual cucumbers-in-seats – perhaps due to the absurd contrasts required – but shifting to a 'cartoon' style with the prompt "editorial cartoon of the King of Belgium giving a speech to many cheering cucumbers, professional illustrator" got a lot closer:

https://labs.openai.com/s/4IonSKYkl0okhNvzJmEAH30K (good)

https://labs.openai.com/s/ZeadCzZ9WqeASYXPlOb13wDV (good)

https://labs.openai.com/s/nERf6bALKEBsQQBvPVsAH7o4 (good)

https://labs.openai.com/s/7dsZu3bwtfGZZxxJYtg9lTVf

If I had more time & credits to burn, I suspect working off those could eventually hit something really apt... but it takes some work & tinkering.

OP did say what prompt they used

I've often seen people show off their autogenerated images and report only approximate paraphrases of their actual prompts.

There's one screenshot showing the prompt – but in $CURRENT_YEAR, I view all screenshots with at least a little suspicion, especially when there was a way to highlight the pseuod-watermarked image – OpenAI's native 'Share' – that would've provided stronger proof, direct from OpenAI, of exactly the prompt associated with an image. Hoaxes are everywhere! I've added DALL-E bottom-right color-squares to non-DALL-E images, & seen others do the same, as a subtle joke.

So I generally believe the OP, but don't rule-out the possibility there's been tampering to make some point.

It definitely requires some very detailed descriptions and sifting through to find a good one. One time I've regenerated a prompt as well because the existing 4 were just not that good. But I did get some great ones at a pretty good usable:unusable ratio.

I have access. You get for trials for each query. I have to say that usually there is only one that is good on those three. Sometimes you need to refine your query. I'm pretty impressed as a user.

They can definitely be that bad quite frequently. I've actually been a lot happier with stable diffusion outputs lately (doesn't hurt that they're free too).