I invite you to actively start photographing or videotaping some strangers if you think such behavior is entirely within people's expectations.
http://www.youtube.com/watch?v=ym7x7twSoqc
Some of them he is clearly in a school or something, which may not be public. In others he is outside.
I believe this is how you expected people to react?
Edit: found it. https://news.ycombinator.com/item?id=4739152
I can sue you for using my likeness depending on how you utilize your footage.
You certainly don't have carte blanche to use images of me you took in public.
But even if you publish it (in Facebook for example) you are protected by some "fair game" clauses, just as you have the right to request that particular photo to be taken down. Either way nothing is absolute and as you said, it gets murky pretty fast.
Just look at all the photos you have taken, how many strangers are in them?