Re:OpenAI's justification for why training data is fair use,...
by denton-scratch on Thursday October 05, 2023 @05:52PM
(#37784745)
Yeah, I said that.
You're not the only commenter that has pointed out to me that the model doesn't contain the training corpus; that the model is analogous to a human writer who learns to write by becoming a voracious reader.
I acknowledge that, I think the analogy is fair. The problem is that the memory capacity of the model appears to be humungous; I've never played with ChatGPT, but I understand that it can quote accurately and at some length from a wide range of literature.
By the model, I mean the model behind ChatGPT. I suspect this "memory" amounts to having unimaginable numbers of parameters, i.e. something like overfit. I realize that's just one LLM, and there are other models that are neither generative nor language models, and that have to be much more restrained in what they consider a "parameter".
You're not the only commenter that has pointed out to me that the model doesn't contain the training corpus; that the model is analogous to a human writer who learns to write by becoming a voracious reader.
I acknowledge that, I think the analogy is fair. The problem is that the memory capacity of the model appears to be humungous; I've never played with ChatGPT, but I understand that it can quote accurately and at some length from a wide range of literature.
By the model, I mean the model behind ChatGPT. I suspect this "memory" amounts to having unimaginable numbers of parameters, i.e. something like overfit. I realize that's just one LLM, and there are other models that are neither generative nor language models, and that have to be much more restrained in what they consider a "parameter".