Re:Ask HN: Will reliance on GPT result in a local maximum of...
by AnimalMuppet on Thursday March 30, 2023 @08:35AM
(#35371668)
I worry about GPT's training set becoming diluted. It's trained with a snapshot of human... not knowledge, exactly. Human text, which contains much knowledge but also some mistakes, some propaganda, and some lies.
Then GPT produces a huge amount of new output, which reproduces human knowledge, mistakes, propaganda, and lies, and adds new mistakes of its own. And from that point on, you have a choice to make. Do you use GPT output to train GPTs, nor not?
If yes, you risk a GPT echo chamber, where GPTs are increasingly trained on GPT output, and human-written text becomes a smaller and smaller fraction of the input. (And some of the GPT input becomes second- or third- generation GPT output.) I have a hard time seeing that increasing the correctness of the output.
If you exclude GPT output from the training data, though, how do you make that work? Do you freeze the training data to what existed in 2020, say? That cuts GPT off from all new knowledge. But if you don't do that, you have to be able to reliably filter out GPT-generated text. How are you going to do that?
Then GPT produces a huge amount of new output, which reproduces human knowledge, mistakes, propaganda, and lies, and adds new mistakes of its own. And from that point on, you have a choice to make. Do you use GPT output to train GPTs, nor not?
If yes, you risk a GPT echo chamber, where GPTs are increasingly trained on GPT output, and human-written text becomes a smaller and smaller fraction of the input. (And some of the GPT input becomes second- or third- generation GPT output.) I have a hard time seeing that increasing the correctness of the output.
If you exclude GPT output from the training data, though, how do you make that work? Do you freeze the training data to what existed in 2020, say? That cuts GPT off from all new knowledge. But if you don't do that, you have to be able to reliably filter out GPT-generated text. How are you going to do that?
It's a problem. I don't see a clear solution.