Re:DALL-E 3: Improving image generation with better captions...
by Jackson__ on Friday October 20, 2023 @02:22AM
(#37952911)
Well, not quite. The linked hf space uses the ViT-H-14 OpenCLIP model, which was trained on the Laion-2B dataset[0], which I'd categorize as fitting the reports description of "noisy and inaccurate image captions" perfectly.
Re:DALL-E 3: Improving image generation with better captions...
by aargh_aargh on Friday October 20, 2023 @02:34AM
(#37952992)
I see, so they relabelled a smaller (more representative) subset for the captioner manually but more diligently, then used that to relabel their large set (analogous to LAION) with more descriptive captions, then trained DALL-E 3 on that.
By the way, in the case of CLIP-Interrogator-2, I wonder how they came up with their wordlists (included in the repo). Are they just all unique terms from OpenCLIP?
https://huggingface.co/spaces/fffiloni/CLIP-Interrogator-2