Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by scottlegrand2 on Thursday April 26, 2018 @02:09PM (#16934157)

What they're not saying is that one can't use all nvlink bandwidth for gradient reduction on a DGX-1V with only 4 GPUs because nvlink is composed of 2 8-node rings. And given the data parallel nature of this benchmark, I'm very interested in where time was spent on each architecture.

That said, they fixed this on NVSwitch so it's just another HW hiccup like int8 was on Pascal.

[ Parent | on HN ]

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by elmarhaussmann on Thursday April 26, 2018 @10:13PM (#16937961)

For this benchmark, NVLink and gradient reduction isn't the bottleneck. The performance scales almost perfectly linearly from one GPU to four.

[ Parent | on HN ]