So, as to 'they chose this themselves', that's begging the question a little - as we've already seen I think that believing in determinism doesn't stop you from believing that people can choose things. As to 'they deserve it', questions of what people deserve is a huge other topic, and I think it's most likely possible to define desert in terms that fit within a deterministic viewpoint.
However, perhaps you're right about its effect since I do in fact think that vengeance is a thoroughly dumb rationale for treating people badly. This is to such an extent that I'd feel surprised to find anyone seriously try to justify it, which is why I didn't really engage with your apparent suggestion that it was an important justification. There are lots of ways you can try to justify criminal punishment in my view but 'vengeance' doesn't fit into any moral system I consider valuable unless it's actually being used as a code for game theoretic concerns or rehabilitation.
But I don't feel so strange thinking this, since most people I know think the same, even those who don't share my views on free will and I'm pretty sure I believed it before I came to that conclusion on free will.
I mean, a key example of how vengeance still plays into punishment is the level of support for capital punishment.
I'm glad we find it equally unjustified.
Also, I agree that you certainly can dismiss vengeance as a valid argument without dismissing free will, btw. - a lot of people who do believe in some variant of free will absolutely do reject vengeance. My point was more to offer up an example of something that is very hard to reconcile with rejecting free will.
I agree that there are still people who needs to be locked up, including under possibly brutal sentencing terms. E.g. Anders Behring Breivik who carried out the 2011 terrorist attacks in Norway is up on a parole hearing today - if he gets out (it'd take a miracle) it doesn't matter if it's free will (in any sense of the term) or purely deterministic processes that leads him to act if he is still probabilistically likely to carry out another attack - it's still more important to keep the public safe from him than to worry about whether or not he had a genuine choice.
To the extent there's a measurable effect of seeing the punishment of others, that too might be a justification, though I find that dicier. But if that is done based on a cold calculation of how it affects offending rates that to me is still very different from vengeance.
So to be clear I'm not arguing it removes the justifications for punishments, and I agree with your mention of seeing it as a game theoretic situation - even really harsh punishments might be the best choice in some circumstances. But it removes "they chose this themselves" or "they deserve it" as a justification (the feeling is still valid; acting on it becomes cruel), and that makes for a very different legal system. It also shifts a focus towards considering that if it's not about vengeance, then you should look at how to actually optimise the balance between the public good and treating the prisoners as humanely as you can without reducing the public benefit of the punishment too much.
Norway is far from achieving that, but if you look at e.g. Bastøy fengsel [1] ([2] has links to assorted longer articles), you can get one idea of what de-emphasising vengeance and taking rehabilitation and trying to address reoffending rates might start to look like (though Norway has by no means solved that, Bastøy also certainly haven't made it worse, but it also is not the norm in Norway either - yet anyway). For some people, it might well be that harsher prisons works better, or are necessary (I doubt Breivik will ever get the chance to serve his time somewhere like Bastøy for example). (Bastøy is the - so far - culmination of a process of reform that started in the 1960's, at a point where Norwegian prisons were quite brutal and the brutality of it eventually triggered a strong counter-reaction.)
For others even less prison (Norway as it is have sentences that would look comically short for e.g. Americans or British people). But evaluating it as a "game theoretic situation" as you say would in any case very likely lead to very major change to how the legal system would work most places in the world (including Norway)
[1] http://bastoyfengsel.no/English/
[2] https://en.wikipedia.org/wiki/Bast%C3%B8y_Prison