I know more than a few people in my small town who got arrested for burglaries and robberies. I know a lot of victims too. Rape basically isn't reported because if the neighbor's boy gets in trouble, that'll ruin the relations between half the town. There are the town drunks who got pulled over more times than you can count and eventually killed someone in a car accident.
You're passing by hundreds if not thousands of people in a few hours in a city. A methhead dancing naked on the street corner is going to catch your eye. The thousands of normal people won't.
In a small town, you're not seeing anyone at all some days. Maybe you're just passing by 5 cars. The opportunity to witness anything is lower.
Of course, plenty of small towns are nice and safe. But the ones that have any problems are proportionally going to be way worse, due to lower population.
As an adult I realize why they always wanted me to stay over was he wouldn't lay into them while I was there. He'd just leave and go to brothers house. Or as we found out later, his girlfriends.