If the datasets were well curated, we would sample directly from the learned distribution instead of using high classifier-free guidance but by extracting image-text-pairs from the Common Crawl web data dump you're not really creating a well curated dataset.