You haven't accidentally sent a prompt to a locally-running LLM using the wrong delimiters/formatting then. Noise, random parts of words, looping repetition, it all happens.

Sure, I have no idea what you're running locally, just assume I'm talking about frontier models. It doesn't matter to the argument, because the "stochastic parrot" claim --- here and in Gebru's paper --- is categorical.