News for hackers. Stuff that matters.
Re:Train TensorFlow models faster and at lower cost on Cloud... by antognini on Wednesday December 12, 2018 @06:50PM (#18668708)
I worked on a research project that set out to answer that question:

https://arxiv.org/abs/1811.03600

We looked at a bunch of different model architectures and datasets and found that you can get speedups from larger batch sizes up until a certain point, but that point is different for different datasets and architectures. The range of good hyperparameters is also narrower for larger batch sizes, which makes tuning harder.

Re:Train TensorFlow models faster and at lower cost on Cloud... by gdahl on Wednesday December 12, 2018 @09:28PM (#18669441)
To clarify, we found the range shrink if one trains for a fixed number of epochs and expand if one trains for a fixed number of steps. So tuning can be easier or harder depending on the budget.