I personally subscribe to what I view as a more logically consistent version where the only thing that matter is how confident we are that someone rejects the social contract. If e.g. they repeatedly keep committing clear-cut crimes against clear-cut victims, they obviously think it doesn't apply to them, and in my view they literally should have no more human rights than a rock. That should apply whether they are a CEO defrauding customers, or a homeless person stealing a bike.
However, most people think it's a proportional/gradual kind of thing, hence e.g. jail.
I think the problem is actually uncertainty. If someone robbed a gas station, it's not that they broke the social contract N% of the way and are N% "pond scum"; it's that we are about N% sure that they are 100% "pond scum". So, before we are sure enough, they have full rights and we should try to re-integrate them and not punish them beyond fixing the damage, and after we are sure enough (which may be on the 5th bike, 3rd gas station, or the 1st rape), they have no rights and yeah, it's e.g. mines/lethal injection.
Either way it responds to the original objection... punishment/retaliation works fine with negative-only rights.
I guess in that case we have to assume we have very few actual rights. Being jailed for a crime is the removal of many of what people would consider fundamental rights (eg. the right to travel freely).