I don't think that'd hold up against a legal review. It seems like an unreasonable position that some neighbourhood is so terrible that standing there for 20 minutes is an imminent threat. It might even be true, but that isn't a baseline a judge should really accept. The residents who live there obviously get through the day.

It may well have been very dangerous, but realistically it is hard to make dropping someone off in a residential area a crime. Threatening a driver with physical violence is definitely a crime though.