This study by the NYU School of Law figures that 4 in 10 (!) US prisoners are either non-violent or have already completed long sentences, and can be set free with minimal risk.

https://time.com/4596081/incarceration-report/

That said, I agree that reforming drug policy in particular would also be a great way to reduce mass incarceration.