Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on What is the difference between deep learning and usual machine learning?. Parent comment
The rectified linear unit also converges about 75% faster than the classic Sigmoid or Tanh. the ReLu is F(x) = max(0, x) so the gradient propogated is 1 rather than a more "curvy" value.
There are no comments on this comment yet.
The rectified linear unit also converges about 75% faster than the classic Sigmoid or Tanh. the ReLu is F(x) = max(0, x) so the gradient propogated is 1 rather than a more "curvy" value.