The point is some places in some cities at some times carry high risk to whoever drives there. I consider it immoral for the company to force an employee (or contractor) to go to such high-risk place, and for that person I think makes complete sense to refuse going to such place and risk their assets (e.g. car) and their health for a relatively small payment.

Portland, Seattle and Chicago are examples of the cities which had recently well-documented examples of some areas being dangerous and carrying risk of violent encounters.