News for hackers. Stuff that matters.

A comment on Deep Learning Benchmarks. Parent comment

Re:Deep Learning Benchmarks by hobofan on Monday March 07, 2016 @07:14AM (#11238438)
Not exactly, no. But not even the output of the same framework will always be the same since you are usually randomly initalizing the weights in a network and randomly picking the samples used in SGD[1] (the seed for the RNG could of course be a fixed one to mitigate this somewhat).

But in the end, if you are using the same model, the same solver and the same RNG, yes the output of all frameworks should be the same. In practice this also mostly holds true, since the stochastic processes involved are geared towards finding a good local minimum, which is the same given a model and a dataset.

[1]: https://en.wikipedia.org/wiki/Stochastic_gradient_descent

[ Parent | on HN ]
Re:Deep Learning Benchmarks by reality_hacker on Monday March 07, 2016 @01:00PM (#11240323)
So, there is a strong chance that you are comparing very different things, and frameworks which perform not good in terms of speed can be far superior in terms of models quality.
[ Parent | on HN ]