News for hackers. Stuff that matters.
Re:DALL-E 3: Improving image generation with better captions... by alievk on Thursday October 19, 2023 @04:59PM (#37948649)
Core takeaways:

- major improvement is from detailed image captioning

- they trained an image captioning model to produce short and detailed captions

- they use T5 text encoder

- they use GPT-4 to "upsample" short user prompts

- they train a U-net decoder and distill it to 2 denoising steps

- text rendering is still unreliable, they believe the model has hard time mapping word tokens into letters in image

Re:DALL-E 3: Improving image generation with better captions... by GaggiX on Thursday October 19, 2023 @06:36PM (#37949727)
They have only distilled the diffusion model used as a replacement of the VAE decoder, not the main diffusion model.
Re:DALL-E 3: Improving image generation with better captions... by alievk on Friday October 20, 2023 @05:34PM (#37961818)
So the latent to pixel space conversion is made with another diffusion model?

This is kind of refiner used in SDXL?

Re:DALL-E 3: Improving image generation with better captions... by GaggiX on Thursday October 26, 2023 @01:14AM (#38021850)
>So the latent to pixel space conversion is made with another diffusion model?

Yup

>This is kind of refiner used in SDXL?

The refiner only works on the latent.