Yup, variety is a big part of it. You don't just want a large language model that can produce meticulous text, you also need to be able to handle user input, which will frequently include faulty input itself. Otherwise you end up with something as mechanic and inflexible as google assist. Interactions with large language models can be much more organic if you have this kind of input data.

Agreed. We can make infinite grammatically correct text with perfect spelling, but if we limit the training set to that alone then ppl who wright liek this cant uze the ai lol & will be all lije "wtf m8 this aint skool i don care bout spekling rite u know wot i ment y cant ur machine?!?"

…

and while I enjoy my ivory tower as much as the rest of us, I do think it's important for the AI to understand everyone, not just me.