Most use common crawl, and most seem to upweight the scientific publication part of it (from what I gather speaking to folks who train LLM models).