Committing crime is a crime against society and thus yourself, or something like that.

That's one view of justice anyway. I'm more inclined towards crimes being against specific persons or groups of distinct persons, in which case your thesis would be correct, but it's a minority opinion.