Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by twtw on Thursday April 26, 2018 @10:47AM (#16932317)

Great work, RiseML. This benchmark is sincerely appreciated.

I wonder whether NVLink would make any difference for Resnet-50. Does anyone know whether these implementations require any inter-GPU communication?

[ Parent | on HN ]

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by elmarhaussmann on Thursday April 26, 2018 @10:54AM (#16932392)

They don't require it but some of the ResNet-50 implementations can make use of it (e.g., the ones in the Docker containers on the Nvidia GPU Cloud). But even the ones without seem to scale to 4 GPUs pretty well. This may be a different story for 8 GPUs and larger/deeper networks, e.g., ResNet-152.

[ Parent | on HN ]