I think that for smaller crimes it would be good to randomly prosecute, say, 5% of all instances of that crime (assuming a truly random choice is made). This stops career thieves from getting away with things forever but reduces the burden on society to prosecute every instance.