In this case, you cannot see the statement as being about boolean true/false values. It only works on an ordinal scale, and our calibration of the cutoff for "wrongness". Slavery here is assumed axiomatically to be "really really bad". In fact, among all the bad things, it is the worst. But is it wrong? That depends on how bad something must be to be wrong. But if you do not consider slavery to be wrong, but merely bad, nothing can be wrong, because everything is less bad than slavery.

So this is basically Lincoln trying to argue for AUC as a better metric than sensitivity/specificity.

(Bear with me, it's been a while since I've done rigorous proofs.)

Let's say there's a function R that outputs a numerical value of an actions morality.

Let's define an action as wrong if R(action) is below some threshold M.

So what Lincoln is saying is:

R(slavery) >= M -> R(action) >= M for all actions

If we assume Lincoln's statement to be true, that must mean that R(slavery) <= R(action) for all actions.

Which can be reduced to R(slavery) = min(R(actions))

Translated to English, Lincoln is saying that slavery is most morally despicable act.