Re:Facebook Trains ImageNet in 1 Hour
by jamesblonde on Tuesday June 13, 2017 @08:27AM
(#14544247)
I disagree that this is not an architecture for others. We are a research lab, and we're building a distributed cluster. Cheap infiniband. Lots of gtx1080Tis. It's not that expensive to have a 40-GPU cluster with 10 4U servers (about 100K Euro).
[ Parent
| on HN ]
Re:Facebook Trains ImageNet in 1 Hour
by gm-conspiracy on Tuesday June 13, 2017 @08:46AM
(#14544329)
What mobo/CPU are you using? How much RAM in each server?
Can you provide some more detailed specs?
Thanks!
Re:Facebook Trains ImageNet in 1 Hour
by jamesblonde on Tuesday June 13, 2017 @11:37AM
(#14545577)
CPU is not that important.
A motherboard that has PCIe 3.0 and 7 PCIx16 slots is ok for at least 3 GPUs. Each GPU takes typically two slots. Then, a >1400 Watts PSU.
Here is a relatively cheap box that also has 8 disk slots (for hadoop):
https://exxactcorp.com/index.php/solution/solu_detail/320
Some people are using water cooling setups to get 7 GPU's on a 7 PCI slot MB. You need to cut off the DVI that would otherwise occupy another slot, but overall it's not that time consuming and reduces your networking needs.
Re:Facebook Trains ImageNet in 1 Hour
by jamesblonde on Tuesday June 13, 2017 @11:47AM
(#14545673)
You can see the potential scale-out you can get for Tensorflow here on this PR:
Are you going to rent access to your cluster? :)
Re:Facebook Trains ImageNet in 1 Hour
by jamesblonde on Tuesday June 13, 2017 @11:39AM
(#14545592)
To users in Sweden at www.hops.site
It provides Jupyter/Hadoop/Spark/Tensorflow and GPUs (as a resource in YARN).