News for hackers. Stuff that matters.

A comment on GradientFlow: Training ImageNet in 1.5 Minutes on 512 GPUs. Parent comment

Re:GradientFlow: Training ImageNet in 1.5 Minutes on 512 GPUs by JustFinishedBSG on Monday February 25, 2019 @04:12PM (#19249495)
We can't even do that, the biggest GPUs today are at the reticle limits of foundries> They literally can't be made bigger. Of course we can still go the chiplet way but...
[ Parent | on HN ]
Re:GradientFlow: Training ImageNet in 1.5 Minutes on 512 GPUs by frankchn on Monday February 25, 2019 @04:30PM (#19249639)
... but power requirements would put a limit to that as well. The RTX 2080 Ti already has a 250W TDP. Putting a couple of those on a single card and you are looking at >1000 W. Cooling and power becomes very hard as we are effectively trying to run and cool a space heater at the same time.
[ Parent | on HN ]
Re:GradientFlow: Training ImageNet in 1.5 Minutes on 512 GPUs by CyberDildonics on Monday February 25, 2019 @04:28PM (#19249617)
But what?
[ Parent | on HN ]