News for hackers. Stuff that matters.
Re:DALL-E 3: Improving image generation with better captions... by Jackson__ on Friday October 20, 2023 @02:22AM (#37952911)
Well, not quite. The linked hf space uses the ViT-H-14 OpenCLIP model, which was trained on the Laion-2B dataset[0], which I'd categorize as fitting the reports description of "noisy and inaccurate image captions" perfectly.

[0] https://laion.ai/blog/large-openclip/

Re:DALL-E 3: Improving image generation with better captions... by aargh_aargh on Friday October 20, 2023 @02:34AM (#37952992)
I see, so they relabelled a smaller (more representative) subset for the captioner manually but more diligently, then used that to relabel their large set (analogous to LAION) with more descriptive captions, then trained DALL-E 3 on that.

By the way, in the case of CLIP-Interrogator-2, I wonder how they came up with their wordlists (included in the repo). Are they just all unique terms from OpenCLIP?