>And now that AI generated content is everywhere, that means it's in the next training dataset. Except you can't train AI on AI generated content (https://arxiv.org/pdf/2307.01850.pdf). And no, you can't reliably detect AI generated content either (https://twitter.com/AtoosaChegini/status/1709738910165635364).

This is the big one. Every single source of data post March 2023 is corrupted beyond repair, and it's only going to get worse.