Non-violent offenders still mess people's lives up and will likely continue to do so given opportunity. Imagine losing your retirement savings due to being conned. The so-called victimless crimes (eg: drug use), are the ones which could use the most overhaul.
Even in the case of violent crime, the necessity of imprisoning them for as long as the US does, is questionable. Again in Europe jail sentences tend to be much shorter.