GradientFlow: Training ImageNet in 1.5 Minutes on 512 GPUs

Re:GradientFlow: Training ImageNet in 1.5 Minutes on 512 GPUs by WrtCdEvrydy on Monday February 25, 2019 @03:33PM (#19249180)

For reference, 10 years ago, graphics cards were on a 65nm platform while today, we use 14nm graphics cards.

The physical limitations may be reached, but we'll just throw more cores unto the PCB to compensate.

[ Parent | on HN ]

Re:GradientFlow: Training ImageNet in 1.5 Minutes on 512 GPUs by JustFinishedBSG on Monday February 25, 2019 @04:12PM (#19249495)

We can't even do that, the biggest GPUs today are at the reticle limits of foundries> They literally can't be made bigger. Of course we can still go the chiplet way but...

[ Parent | on HN ]

Re:GradientFlow: Training ImageNet in 1.5 Minutes on 512 GPUs by frankchn on Monday February 25, 2019 @04:30PM (#19249639)

... but power requirements would put a limit to that as well. The RTX 2080 Ti already has a 250W TDP. Putting a couple of those on a single card and you are looking at >1000 W. Cooling and power becomes very hard as we are effectively trying to run and cool a space heater at the same time.

[ Parent | on HN ]

Re:GradientFlow: Training ImageNet in 1.5 Minutes on 512 GPUs by CyberDildonics on Monday February 25, 2019 @04:28PM (#19249617)

But what?

[ Parent | on HN ]