News for hackers. Stuff that matters.
Re:Build a fast deep learning machine for under $1K by web007 on Thursday February 09, 2017 @11:28AM (#13608073)
Are multi-GPU systems worth it until you have a TON of experience building parallel models? Almost all of the reference architectures I've seen are essentially serial, and TF doesn't have good data-parallelism built in to make use of N GPUs, N > 1.
Re:Build a fast deep learning machine for under $1K by tripzilch on Friday February 10, 2017 @10:41AM (#13616082)
Maybe this is a stupid idea, but if you had two GPUs, doesn't that basically mean, in terms of good parallelism, that at least you can train two models/networks at the same time, roughly?

Because as long as you're in the research and development phase, that'll help cut your coding/training/testing/adjusting cycle. I assume you'll be tuning your hyperparameters, perhaps on somewhat smaller test models, but they will still take half a day or so to train? That means you can try (almost) twice as many hyperparameter configurations in the same time. It still helps to spin up a second test with a different selection of parameters, even if you haven't gotten the results back from the first test, right?