News for hackers. Stuff that matters.
Re:The Sigmoid Function in Logistic Regression by Animats on Tuesday May 17, 2016 @11:22AM (#11714394)
Sigmoid functions also come up in neural nets. But that's because you need something that's differentiable. If you just used a step function, backpropagation wouldn't work, because small changes in the tuning parameters wouldn't do anything. You can think of a sigmoid as a step function pushed through a low-pass filter.

It would be interesting to try step functions run through low-pass filters with different cutoffs to see which work best for logistic regression. Anything that produces a monotonically increasing value ought to converge. The only question is how fast.

Re:The Sigmoid Function in Logistic Regression by eggy on Tuesday May 17, 2016 @12:40PM (#11715050)
I was going to chime in about neural nets too, since I thought that was what the article was going to mention instead of just defining it for logistic regression. It is used because it is continuous and differentiable as you say, and it keeps the bounds neat between 0 and 1, although most modern deep neural nets use an ReLU, or rectifier function of some sort. The softplus is an ReLU, that has the nice property of its derivative being the sigmoid function.
Re:The Sigmoid Function in Logistic Regression by argonaut on Tuesday May 17, 2016 @03:29PM (#11716578)
To expand on what gcp is saying, that's not true. Rectified linear units do better than sigmoids but are not differentiable (but are sub-differentiable), and ReLUs have the problem where small changes don't do anything (in the negative regime).
Re:The Sigmoid Function in Logistic Regression by gcp on Tuesday May 17, 2016 @12:23PM (#11714889)
>backpropagation wouldn't work, because small changes in the tuning parameters wouldn't do anything

cough Rectified Linear Units cough