You don't have to be "bad" to be disliked, you just have to be distant and distrustful. In that neighborhood it's not irrational to build a fortress in your business, but that's exactly why this one diner stands out, he's one of them and he lives with them in his business.

You probably couldn't go in and safely start a business there, because of the terrible reality of the neighborhood, and because you're not part of the neighborhood. It's plausible that a local could do it.