But before its concern about a monoculture, the article 1st points out that mere prediction-training (on another model's outputs or fresh data) can't truly match RLHF in instilling some desired behaviors. And that presents a bit of a tension with the article's 2nd concern: if mere output-mimicking can't match more-sophisticated training, then it can't really call-into-being the excessive uniformity, either.
And maybe: the dose makes the poison. A little cribbing might be a beneficial partial accelerator for smaller teams & newer projects, even if a lot is ineffective at ever fully replicate OpenAI model behaviors, or deleterious if effective-and-overdone.
So the article isn't really a strong case for not trying this at all – just for keeping the potential limits & downsides in mind, in any experiments with this technique.
The reasoning that the results won't be quite as good as RLHF, or result in a perfect 'clone' of ChatGPT's capabilities, seems pretty good to me.
But the idea it won't be helpful at all, especially to projects that are just seeking some incremental advantage? Seems speculative.
In particular, when you read the linked comments from Yoav Go, he outlines a potential RL process that uses automated scoring for non-exact similarity to preferred answers. Using known (or even 'probably') good answers from ChatGPT output, as the inputs to that process, seems like it could often offer some of the same sort of improvement to other models as ChatGPT obtained via its RLHF.