Ideology often gets in the way of providing the best solution to the greatest amount of people. I personally don't see much value in keeping a broken half-market like health insurance putting along in the name of "free markets". That just makes people blame markets for the failures of the public/private arrangements, which hurts them in other areas where it does make sense, where markets should be the default.
Others like private prisons and private military contractors makes the public lose trust in public institutions. Having a trustworthy justice and government system is critical for all markets to succeed.
Additionally a harder public/private distinction will put greater weight in these decisions and more responsibility. It's very easy for a politician to grab the half-measure public/private mix while pretending to appease both sides.
FWIW, one should also consider that in a Libertarian system FAR fewer people would be in jail in the first place - perhaps as few as none, since Libertarian ideology is not big on forced imprisonment in the first place. The typical Libertarian view of justice is rooted more in the idea of restitution for damages caused (where applicable) than "punishment".
On the right you've got some varieties of anarcho-capitalism that support systems of polycentric law that emerge from markets for law enforcement and criminal justice. On the left you've got anarchist movements like anarcho-syndicalism which propose a decentralized or federated system of social institutions that would be determined by direct action and direct democracy.
I think libertarians would benefit from creating a clearer spectrum within their own community between the more practical (and popular) small government/minarchist side and the more extreme anarcho-capitalist privatize everything group. Too often they seem to be happy having these lines blurred. Plus I don’t think there is any contradictions or significant compromises in doing so. If anything pushing for pipe dream imaginary economic systems or half solution public/private stop gaps isn’t helping anyone.
I also made no claims regarding the viability or merits of any of the viewpoints I was describing. That's not a conversation I'm interested in having in this context.
Besides, most of these sorts of private/public relationships are unlike other true markets anyway with very little competition and only one customer to appease. We end up with a system with even more opaque responsibility/liability chains and encourage crony business deals.
Without any discernible benefit for the private nature and creating negative incentives that encourage more violence by correlating the success of an industry on the greater imprisonment of people, I can’t see any net-benefit for them to exist.
I tend to be a fan of markets in a practical sense but I wish we had clearer distinctions between these public/private operations that are spun as being more “efficient”. Plus a ton of people use these failures of public/private organizations to disparage markets. So it’s not doing anyone any favours to support these half baked pseudo market arrangements, even if you are pro market. In most situations we need purely public or market solutions. I’ve seen few examples of mixing the two that left us better off (the US health insurance ‘market’ is the shining example).
[1] as opposed to the more fringe absolutist anarcho-capitalist stuff ala Mises which does push for private everything