News for hackers. Stuff that matters.

A comment on Standardizing OpenAI’s deep learning framework on PyTorch. Parent comment

Re:Standardizing OpenAI’s deep learning framework on PyTorch by nil-sec on Saturday February 01, 2020 @10:08AM (#22210848)
So you are saying the system memory is 300GB and you can train your model on the cpu instead? Well yeah you can always do that but training will be slow because your model is not trained on the GPU. What’s the point?
[ Parent | on HN ]
Re:Standardizing OpenAI’s deep learning framework on PyTorch by sillysaurusx on Monday February 03, 2020 @12:59PM (#22226897)
It's not that slow. And you can use many TPUs together to make up the speed difference.
[ Parent | on HN ]
Re:Standardizing OpenAI’s deep learning framework on PyTorch by nil-sec on Monday February 03, 2020 @03:32PM (#22228670)
If that were the case I am wondering why anyone would buy GPUs? I invite you to retrain a state of the art model of your choice on a CPU and see how far you get.
[ Parent | on HN ]
Re:Standardizing OpenAI’s deep learning framework on PyTorch by sillysaurusx on Monday February 03, 2020 @07:15PM (#22230899)
We fine-tuned GPT-2 1.5B for subreddit simulator using this technique. https://www.reddit.com/r/SubSimulatorGPT2Meta/comments/entfg...
[ Parent | on HN ]