Re:Standardizing OpenAI’s deep learning framework on PyTorch
by sillysaurusx on Thursday January 30, 2020 @01:56PM
(#22194178)
I'll bet you $20 that GPT-3 will be in Pytorch.
My point is that we probably won't be seeing more awesome projects from OpenAI written in Tensorflow. And that's unfortunate. Memory saving gradients were particularly helpful.
Re:Standardizing OpenAI’s deep learning framework on PyTorch
by chillee on Thursday January 30, 2020 @02:01PM
(#22194238)
I mean, this starts to sound like your complaint isn't that OpenAI is standardizing as a framework but that the framework is PyTorch :)
Re:Standardizing OpenAI’s deep learning framework on PyTorch
by SpaceManNabs on Thursday January 30, 2020 @03:17PM
(#22195063)
That is usually the case with framework wars.
Re:Standardizing OpenAI’s deep learning framework on PyTorch
by octbash on Thursday January 30, 2020 @07:42PM
(#22197738)
My counter-arguments (as a huge PyTorch fan) are:
1. GPT hasn't really been about model/architectural experimentation, just scale. GPT-2 and GPT were architecturally very similar. Scale, especially at the scale of GPT-*, is one avenue that TensorFlow does have an edge over PyTorch 2. Work on GPT-3 probably started quite a while ago.
OpenAI has researchers, and it has people who work on infrastructure for the researchers. Researchers are free to use whatever they want, but if the infrastructure developers want to build something for the researchers, it's beneficial to have a "standard".
OpenAI researchers will still be able to use other frameworks for their own research. All this means is that their major infrastructural projects will be released in PyTorch.