News for hackers. Stuff that matters.
Re:Standardizing OpenAI’s deep learning framework on PyTorch by sbrother on Thursday January 30, 2020 @01:27PM (#22193822)
Is there any equivalent of TF Serving for PyTorch? We have been thrilled with how robust and easy it is to deploy our models to production on the TF stack, and it worries me that the inertia in the deep learning community seems to be toward PyTorch.
Re:Standardizing OpenAI’s deep learning framework on PyTorch by calebkaiser on Thursday January 30, 2020 @03:14PM (#22195033)
Have you checked out Cortex? It's an open source platform for deploying PyTorch models easily. I wrote an article for the PyTorch blog about it: https://medium.com/pytorch/how-to-build-production-software-...

GitHub: https://github.com/cortexlabs/cortex

Full disclosure/shameless plug: I work on Cortex

Re:Standardizing OpenAI’s deep learning framework on PyTorch by sbrother on Thursday January 30, 2020 @04:54PM (#22196167)
Thanks! I was not aware of this and it looks fantastic. Is it in the roadmap to target GCP, or even just a generic Kubernetes cluster?
Re:Standardizing OpenAI’s deep learning framework on PyTorch by calebkaiser on Thursday January 30, 2020 @06:20PM (#22197072)
GCP is on the immediate short-term roadmap, and we're investigating on-premise, but don't have a firm timeline on it quite (we're still a small team).
Re:Standardizing OpenAI’s deep learning framework on PyTorch by smhx on Thursday January 30, 2020 @03:17PM (#22195070)
There's also fastapi, which is well-respected in Python. Someone wrapped fastapi / rabbitRPC to serve PyTorch models (with auto-batching to increase serving efficiency) in https://github.com/catalyst-team/reaction
Re:Standardizing OpenAI’s deep learning framework on PyTorch by MiroF on Thursday January 30, 2020 @03:45PM (#22195362)
You can convert the checkpoints over for deployment. I agree the serving infra on TF is much better.