News for hackers. Stuff that matters.
Re:Backpropagation algorithm visual explanation by 2bitencryption on Wednesday June 27, 2018 @12:48PM (#17409682)
question about this part:

> f(x) has to be a non-linear function, otherwise the neural network will only be able to learn linear models.

I thought one of the most common functions was relu, which is linear (but cuts off to 0 for x values below 0)

https://en.wikipedia.org/wiki/Rectifier_(neural_networks)

Re:Backpropagation algorithm visual explanation by brchr on Wednesday June 27, 2018 @02:39PM (#17410656)
It’s exactly the cutoff at zero that puts the "kink" into the function which makes ReLU nonlinear. :)