If, as you say, rehabilitation is impossible, punishment would be illegitimate. Because we don’t harm people just to satisfy our thirst for revenge.
It is possible for someone to be rehabilitated on their own as they change, or through someone taking action that is not punishment (explaining the impact of the actions which the person may not have been thinking of, etc).
It's possible to punish someone with no thought as to rehabilitation. Capital punishment is this. The purpose is not to benefit the person in any way, but to benefit society by providing a negative example to others so they don't attempt the same actions (it doesn't matter if you believe it works, this is the purpose it's done).
Otherwise nobody would care whatever a judge decides.