I used to believe in this, but SF proved it wrong. Simplify writing up or catch-and-release has caused the city's crime rate to go through the roof.
Increasing the penalties for murder is an extremely ineffective way to reduce the murder rate.
I think the simplest way to see the difference between punishment and deterrence is in nature.
We learn not to touch a hot stove by getting our hand burnt. It's usually not very painful, but because it happens immediately and every single time we quickly figure out that touching a hot anything is a bad idea. This creates deterrence without a heavy punishment.
Sometimes a heavy punishment can also cause deference by itself though. Eg messing around with mains electricity. You probably won't get shocked if you're careful, but if you do then the damage can be very severe.
Secondly: This doesn’t capture the need to remove people incapable of integrating with society. Rehabilitation does not capture the reality of the situation. US jails contain many people that do not have the mental faculties to resist overt victimization of vulnerable people. There may also be thoughtful people mixed in, but these groups are nothing alike.