> Logistic regression without regularization blows up if your data is linearly separable. This is an issue you don't really encounter with linear regression.
Finding that your data is linearly separable is usually quite important - I do see students mess this up though, to be fair.
If you want frequentist inference, non-penalized logistic regression is the way to go.
(disclaimer, I'm predominantly Bayesian, so with proper priors the point is actually moot for me.)
scikit-learn is an ML library. It's explicitly not a stats library. The assumptions and defaults are correct for ML use cases. GP comment is entirely correct.
If you want to do widespread statistics, use StatsModels which is designed for that.
Yeah but logistic regression is from stats. If you're going to implement ClassicalThingFromNearbyField in an ML library, people are right to complain when the definition doesn't match. Not surprising anyone, there's a lot of overlap between stats and ML.
If I implemented a "SimpleFraction" model in sklearn, it would be useful to use "[# this class] / ([# all classes] + epsilon)" using epsilon to regularize, but that would be a crazy name. Or at least it would be crazy if I used a default other than epsilon=0.
The use case you think about is not what sklearn does. You cant even use it, because it returns no model metrics you need for a specified model structure.
No one uses sklearn for stats, and logistic regression here is really not stats, it is a not a ML estimated conditional expectation, it is simply a consequence of a ML loss function. Ane in that sense, regularization is useful.
People expecting to get a stats model out of sklearn will fail immediately, since it does not even return errors, covariances or statistics.
EDIT: Apparently this is getting downvoted. To explain a bit more clearly: if you know nothing about the data, then your default should be to run with regularization, since you won't get a useful result if the data turns out to be linearly separable. In the case of linear regression, it's often sensible to start with non-regularized fitting, since that won't mess up your coefficients. But logistic regression is quite simply a different beast altogether, and this is covered in any intro on logistic regression you care to consult.