foohis
Re:Gemini 4 Neon
on Wednesday November 18, 2026 @07:21PM (#50554290)
won't help if they also include trash internet text in the training set as well.
Re:Gemini 4 Neon
on Wednesday November 18, 2026 @05:39PM (#50553353)
It's a much, much better writer than literally every other model.
Re:Gemini 4 Neon
on Wednesday November 18, 2026 @05:16PM (#50553034)
Anthropic spent a lot more time editing their pre-training data with higher-quality material. For example, they used actual books for pre-training, because books are generally run through human editors for higher quality. (this is from the copyright lawsuits)
Other companies would use completely raw internet data for pre-training, which is basically trash. Anthropic's secret sauce is that they filtered and edited down their data for only the highest-quality material.
Meanwhile its obvious that companies like Google would use trash data like email and chat logs for pre-training.
This is a classic case of "less is more" and "garbage-in-equals-garbage-out"