Re:Build a fast deep learning machine for under $1K
by tripzilch on Friday February 10, 2017 @10:41AM
(#13616082)
Maybe this is a stupid idea, but if you had two GPUs, doesn't that basically mean, in terms of good parallelism, that at least you can train two models/networks at the same time, roughly?
Because as long as you're in the research and development phase, that'll help cut your coding/training/testing/adjusting cycle. I assume you'll be tuning your hyperparameters, perhaps on somewhat smaller test models, but they will still take half a day or so to train? That means you can try (almost) twice as many hyperparameter configurations in the same time. It still helps to spin up a second test with a different selection of parameters, even if you haven't gotten the results back from the first test, right?
Because as long as you're in the research and development phase, that'll help cut your coding/training/testing/adjusting cycle. I assume you'll be tuning your hyperparameters, perhaps on somewhat smaller test models, but they will still take half a day or so to train? That means you can try (almost) twice as many hyperparameter configurations in the same time. It still helps to spin up a second test with a different selection of parameters, even if you haven't gotten the results back from the first test, right?