In that situation I just cross the street or turn down another street or whatever. One less thing for her to worry about and I avoid being thought of as a potential rapist. I wouldn't say it's a fear-based thing, though. It's more about avoiding a situation which is uncomfortable and awkward for both parties, however fleeting.