Is there really? For a long time I've lived in an area (East Hastings in Vancouver) that people say is similarly "sketchy", but from what I can see, it's not dangerous to passersby—it's instead that there's a lower class involved in drugs and sex work, and they sometimes attack each other (i.e. people they know, who they have a grudge against.) They don't go around attacking random strangers.