In the decades following the riots, the middle class black population followed the white population into the suburbs.

At the same time, the industrial economy of the rust belt collapsed.

Then the auto industry collapsed.

Any one of these things could set in motion a vicious circle (people leave, local services become underfunded, crime increases, property values decline, more people leave) but Detroit got hit continuously for ~4 decades.