Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin
by killerstorm on Sunday July 19, 2020 @09:29AM
(#23889205)
They commercialize the API. The model is so large you can't run it on a single GPU, AFAIK. OpenAI developed an infrastructure to run the model efficiently. So many companies would rather use API than deploy the model on their own hardware.
They will probably release the baseline model, not model optimized for deployment. There are many optimizations possible such as precision reduction, pruning, distillation, and they don't have to share these optimizations.
> a fast Google search yielded no results and I'm curious.
They will probably release the baseline model, not model optimized for deployment. There are many optimizations possible such as precision reduction, pruning, distillation, and they don't have to share these optimizations.
> a fast Google search yielded no results and I'm curious.
List of pre-trained models on huggingface: https://huggingface.co/models
You can see some of them are prefixed with "google". Also ALBERT is from Google Research.
That's just natural language processing, I dunno what they have in other fields.