We live in a very small city (metro pop 500-600k so maybe 1/4 the size of Nashville and 1/2 the size of Birmingham), and when my wife and I were dating we lived in an apartment downtown. It was great! Only two things made us move.

1. We wanted a big dog, and that's a non-starter for 90% of apartments.

2. Property taxes were 3-4x what they were just a mile or two outside of the city center.