News for hackers. Stuff that matters.
Re:DALL-E 3: Improving image generation with better captions... by gzer0 on Thursday October 19, 2023 @08:55PM (#37950904)
> During testing, we have noticed that this capability is unreliable as words are have missing or extra characters. We suspect this may have to do with the T5 text encoder we used: when the model encounters text in a prompt, it actually sees tokens that represent whole words and must map those to letters in an image.

Ah, interesting remark regarding text rendering.

Re:DALL-E 3: Improving image generation with better captions... by ronsor on Thursday October 19, 2023 @10:09PM (#37951522)
This isn't a new problem. A Google paper from late 2022 mentioned this fact; it went away when they used a byte/character-level text encoder (vs. tokenizing text first).
Re:DALL-E 3: Improving image generation with better captions... by mycall on Thursday October 19, 2023 @10:52PM (#37951876)
It still is an amazing state machine.
Re:DALL-E 3: Improving image generation with better captions... by ollin on Friday October 20, 2023 @12:47AM (#37952471)
"Character-Aware Models Improve Visual Text Rendering" https://arxiv.org/abs/2212.10562 for anyone curious
Re:DALL-E 3: Improving image generation with better captions... by tobr on Friday October 20, 2023 @06:48AM (#37954478)
This is an aside, but in general, aren’t tokens a pretty bad hack to improve context length? I don’t know enough about the theory to say, but it seems like it should make all kinds of things much more fragile, like how well it can understand misspellings, or languages that don’t map well to the tokens available.