alievk

So the latent to pixel space conversion is made with another diffusion model?

This is kind of refiner used in SDXL?

Core takeaways:

- major improvement is from detailed image captioning

- they trained an image captioning model to produce short and detailed captions

- they use T5 text encoder

- they use GPT-4 to "upsample" short user prompts

- they train a U-net decoder and distill it to 2 denoising steps

- text rendering is still unreliable, they believe the model has hard time mapping word tokens into letters in image