Re:OpenGPT-2: We Replicated GPT-2 Because You Can Too
by godelski on Saturday August 24, 2019 @11:07PM
(#20791114)
What I'm saying is that there's going to be a lot of push back for releasing a full trained model because anything with that model can make similarly accurate predictions. Additionally one can fine tune that model and make something better. So someone spends a few million in getting the original model. Someone else spends a few thousand to fine tune a better version.
What I'm saying is there's a huge downside to releasing the full model compared to holding it tight. Many companies don't patent things for similar reasons.
Furthermore, it's a huge field, of which NLP is a significant, but tiny, subset.
The vast majority of models being used in the real world don't generalize thusly, because the data sets and processes they're linked to are bespoke.
The middle ground is the black-box model (perhaps with technical safeguards against decompiling-equivalent, or hosted as a service). It provides the ability to statistically prove bias, while protecting the majority of privileged or private information.