Re:Improving the way neural networks learn
by kghose on Friday August 01, 2014 @01:21PM
(#8121584)
I found that the statement about the cross entropy not true. When y==a the function is non-monotonic with 0 at the extremes but not at the middle. So the "proof" shown is confusing to me.
[ Parent
| on HN ]
Re:Improving the way neural networks learn
by michael_nielsen on Friday August 01, 2014 @01:25PM
(#8121613)
Which statement about cross entropy is confusing?
Hi!
The statement "if the neuron's actual output is close to the desired output, i.e., y=y(x) for all training inputs x, then the cross-entropy will be close to zero"
is not true. The function peaks in the middle (~ 0.7)
Thanks! -Kaushik
Re:Improving the way neural networks learn
by michael_nielsen on Friday August 01, 2014 @03:20PM
(#8122277)
This is addressed in the marginal note attached to the sentence you quoted.
The essential point is that we're considering classification problems, for which the output is intended to be 0 or 1. I address the more general case of regression problems (where y may take any value) in a later exercise.
Hope that helps!