Standardizing OpenAI’s deep learning framework on PyTorch

Re:Standardizing OpenAI’s deep learning framework on PyTorch by Erlich_Bachman on Thursday January 30, 2020 @01:22PM (#22193771)

Or they see this problem and that's why the policy is sanely phrased as follows:

    Going forward we’ll primarily use PyTorch as our 
    deep learning framework but sometimes use other 
    ones when there’s a specific technical reason 
    to do so.

[ Parent | on HN ]

Re:Standardizing OpenAI’s deep learning framework on PyTorch by sillysaurusx on Thursday January 30, 2020 @01:25PM (#22193808)

It never works out this way in practice. You need corporate clout to go against corporate policy. That's the point of having a corporate policy.

Of course they added that caveat. That's probably how this idea got through in the first place. Just point at the caveat and say "But we're not really throwing all the other frameworks under the bus. If everyone decides it's a good idea to use something else, we'll use something else."

Except that likely won't happen, because now as a junior engineer you need to convince N other people that using Jax was a decent choice. And it's against your company's culture to use anything but Pytorch.

This battle of Tensorflow vs Pytorch is bad for everybody involved. OpenAI released a lot of cool and important code related to Tensorflow. They did GPT-2 (tensorflow 1.x), blocksparse (also tensorflow), memory saving gradients (tensorflow 1.x), and now they're announcing they'll likely never be releasing such tooling again. Memory saving gradients have been hugely helpful to us for scaling our models beyond the normal limits.

[ Parent | on HN ]

Re:Standardizing OpenAI’s deep learning framework on PyTorch by gbear605 on Thursday January 30, 2020 @02:04PM (#22194285)

What you’re ignoring is that the switch isn’t from nothing to Pytorch, it’s from Tensorflow to Pytorch. It’s only favoring one library over another. Your scenario with Jax hasn’t changed, and such tooling is going to be released for Pytorch instead of for Tensorflow. I suspect you’re only against this because you prefer Tensorflow to Pytorch.

[ Parent | on HN ]