I was going to chime in about neural nets too, since I thought that was what the article was going to mention instead of just defining it for logistic regression.
It is used because it is continuous and differentiable as you say, and it keeps the bounds neat between 0 and 1, although most modern deep neural nets use an ReLU, or rectifier function of some sort. The softplus is an ReLU, that has the nice property of its derivative being the sigmoid function.
Re:The Sigmoid Function in Logistic Regression
by argonaut on Tuesday May 17, 2016 @03:29PM
(#11716578)
To expand on what gcp is saying, that's not true. Rectified linear units do better than sigmoids but are not differentiable (but are sub-differentiable), and ReLUs have the problem where small changes don't do anything (in the negative regime).
It would be interesting to try step functions run through low-pass filters with different cutoffs to see which work best for logistic regression. Anything that produces a monotonically increasing value ought to converge. The only question is how fast.