Re:Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia ...
by sk0g on Thursday May 14, 2020 @09:56AM
(#23179533)
Mind you, this is an optimised version of BERT, from what I could see in their blog post.
[ Parent
| on HN ]
Re:Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia ...
by shubuZ on Friday May 15, 2020 @12:40PM
(#23194183)
The network topology is the same, no modules are removed, etc. Of course it will be optimized to run on Ampere
Re:Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia ...
by sk0g on Saturday May 16, 2020 @06:00AM
(#23202087)
I'll link the the mention of the optimised versions, but that's not what I mean!
Say earlier model XYZ trained 4 epochs per hour, and BERT trained 2 epochs per hour. Now on a single card if you can train optimised BERT for 4 epochs per hour, that doesn't necessarily mean the same card will handle XYZ at 8 epochs per hour.
It's a technological achievement nonetheless, but the fact that it was heavily optimised for the new architecture, possibly beyond an extent infeasible for non-Nvidia developers, still has to be considered.
[0] https://nvidianews.nvidia.com/news/nvidia-achieves-breakthro...