Yes, this is what it's like. Most of the men seem harmless but when someone starts to follow you it gets scary. This was in a populated area thankfully. A few years ago I attended a women who code meetup and two men decided to try and follow two women home, arguing that it wasn't illegal to walk on the street etc. They had to be threatened with someone calling the police and the women had to delay going home and stay with the group leaving the building before it was clear the men would leave them alone and they could safely walk away.

I would love to see this same experiment repeated in other cities. For some reason this seems very specific to NYC. Has anyone tried this in Los Angeles for example?

It's definitely a thing in San Francisco.