Only 1 in 5 prisoners are there for nonviolent drug crimes. You can't reduce the prison population by more than that unless you want to repeal laws against fraud, burglary, rape, assault, or murder.

One alternative would be to reduce sentences for those crimes, but good luck getting the public to vote for any politician who would propose such a thing.

See https://www.prisonpolicy.org/reports/pie2019.html for some interesting stats on US prisons.