I think in this context, grammatical correctness is irrelevant, and what you actually want is just raw text as written by real humans from all walks of life.

Yup, variety is a big part of it. You don't just want a large language model that can produce meticulous text, you also need to be able to handle user input, which will frequently include faulty input itself. Otherwise you end up with something as mechanic and inflexible as google assist. Interactions with large language models can be much more organic if you have this kind of input data.

Agreed. We can make infinite grammatically correct text with perfect spelling, but if we limit the training set to that alone then ppl who wright liek this cant uze the ai lol & will be all lije "wtf m8 this aint skool i don care bout spekling rite u know wot i ment y cant ur machine?!?"

…

and while I enjoy my ivory tower as much as the rest of us, I do think it's important for the AI to understand everyone, not just me.

Depends on what you are looking to create. Good writing? Then it is an issue. Authentic seeming writing? Then it is exactly what you want.