Madison, however, on first blush seems like an excellent example, in that it's a large (~250k) city ranked highly on most metrics, and has a mid-rise downtown. But a look at a satellite image reveals that the city limits of Madison contain large areas of single-family detached houses, and mid- and high-rises are restricted to the Isthmus and areas immediately surrounding the University [2].
[1] https://en.wikipedia.org/wiki/Philadelphia#Crime
[2] https://www.google.com/maps/place/Madison,+WI/@43.0850975,-8...
Center city is pretty safe. So is South Street. Old City. Plenty of other neighborhoods too numerous to mention with this style of buildings. Yes, crime is relatively high in the city, but it's concentrated in certain areas (see: http://www.trulia.com/real_estate/Philadelphia-Pennsylvania/...). There's a reason the Badlands is called the Badlands and Center City isn't. Judging the whole city to be unsafe by its overall crime stats is ridiculous, unfair, and misleading.
HTH.