Yes, but as time goes on, new data is created that won't be in Common Crawl. Many of the troves of data it archived from in the past are now behind expensive paywalls that are priced with their multibillion dollar clients' access in mind.

If the dataset contains text from 2022, things that happen in 2023 and later won't be in it. The model will only get you so far, and new data, events, concepts, discoveries, etc will be absent.

If we trained models on all text generated up until 1900, for example, you could get it to produce some impressive results if just generating text is the goal. If the goal is to build something that imitates a more general AI, it wouldn't "know" about antibiotic treatments for common illnesses, modern vaccines, either World War, powered flight, transistors, computers, etc. It would only be useful for so much.