News for hackers. Stuff that matters.
Re:Improving the way neural networks learn by michael_nielsen on Friday August 01, 2014 @01:25PM (#8121613)
Which statement about cross entropy is confusing?
Re:Improving the way neural networks learn by kghose on Friday August 01, 2014 @03:15PM (#8122258)
Hi!

The statement "if the neuron's actual output is close to the desired output, i.e., y=y(x) for all training inputs x, then the cross-entropy will be close to zero"

is not true. The function peaks in the middle (~ 0.7)

Thanks! -Kaushik

Re:Improving the way neural networks learn by michael_nielsen on Friday August 01, 2014 @03:20PM (#8122277)
This is addressed in the marginal note attached to the sentence you quoted.

The essential point is that we're considering classification problems, for which the output is intended to be 0 or 1. I address the more general case of regression problems (where y may take any value) in a later exercise.

Hope that helps!

Re:Improving the way neural networks learn by kghose on Friday August 01, 2014 @06:28PM (#8123165)
I see it now, thanks!