You're ignoring deterrence. And yes, deterrence works. I don't park in red zones all that often.
You're also ignoring the benefits of "warehousing". If person A commits an assault a week, general society avoids an assault every week that person A is in jail.
You can argue that the cost exceeds the benefit, but you don't get to ignore the benefit.
> At one point in our history the criminal justice discussions were about how best to rehabilitate the offender rather than punishment.
Yes, that's what they talked about, but what did they accomplish?
Well, let me back up a little first. Criminal penalties as deterrent do work up to a certain point, and there is nothing wrong with embracing them to a civilized and practical degree. But the retributive nature of the American justice system is beyond fetishistic. This is harmful for two reasons.
First, evidence suggests that draconian penalties are counterproductive as a deterrent. The exact psychological reasons for this are being researched, but it is thought that it is a matter of respect for the rule of law. When people perceive a law as unjust and disproportionate, they are more likely to break that law, and that applies to the punishment as well as the crime. It seems paradoxical, but increasing the penalty for a crime beyond what people view as equitable will actually reduce the deterrent effect of the penalty.
Second, we as a society have become so obsessed with retributive sentencing that we will, to that end, actively cut programs that are proven to reduce crime. Any program that helps felons to get legitimate work once they are out of prison is likely to be attacked as "soft on crime" and "wasteful", even though such programs are usually relatively inexpensive and actually save the government money in the long run by reducing recidivism.
It is not hard to understand why giving ex-convicts non-criminal career opportunities is good for society, yet the idea that we should at least throw a couple of bones in that direction is actually quite controversial.