Collecting training data is actually a perfect for crowdsourcing. Images/videos are easier than text, and text is easier than high-quality text, but all are doable.

Only so far as you trust your crowd. It only takes one bad actor to poison an entire data set with a few pieces of copyrighted samples, and now the entire model is subject to potential future litigation.

Wikipedia somehow handles this okay. But yes, there will be work to do to address the issue.

How is it different from stock images? People could sell not their work on the stock sites, and it is pretty hard to detect. Or impossible to detect, if you ask some AI model to repaint the image first, before putting it into the stock.

I feel like the amount of training data needed to compete with commercial models would require a large amount of automation to collect, and it's that same automated collection that platforms recently decided to gatekeep against.

have we exhausted the ways to train models already though? Feeding common crawl into a more advanced training program could result in a better model compared to today

Yes, but as time goes on, new data is created that won't be in Common Crawl. Many of the troves of data it archived from in the past are now behind expensive paywalls that are priced with their multibillion dollar clients' access in mind.

If the dataset contains text from 2022, things that happen in 2023 and later won't be in it. The model will only get you so far, and new data, events, concepts, discoveries, etc will be absent.

If we trained models on all text generated up until 1900, for example, you could get it to produce some impressive results if just generating text is the goal. If the goal is to build something that imitates a more general AI, it wouldn't "know" about antibiotic treatments for common illnesses, modern vaccines, either World War, powered flight, transistors, computers, etc. It would only be useful for so much.