There's a difference between 'fails at the extremes' and 'never worked a little'.

In a society where life is cheap and legal advancement is painful and opportunities are rare, you'll have the situation you mention. If the law is always fatal but rarely applied, why consider it at all?

In a comfortable society that has free food for everyone, a wider range of punishments actually does tend to control crime. In my country Murder can get you (effective) life, and selling marijuana on the street can't. You can buy joints easily but hitmen remain scarce.