Even if it's not as effective as other alternatives, I think it is most likely at least something of a deterrent. For example, I hear of criminals purposely avoiding crimes with more serious punishments (e.g. mugging someone but avoiding hurting or killing them).

In addition, there's a saying that "locks don't keep criminals out, they just keep honest people honest". I think it's likely that punishment at least serves a similar role.

Granted, I think we would probably both agree that the US justice system (for example) is too focused on punishment and not enough on rehabilitation.