The right to privacy in public is generally regarded as a right to casual privacy - in the sense that many people can hear and may capture or accidentally record what you say, but won't publish or broadcast these events.
It seems the problem with Google Glass is more that it's forcing people to consider what has been true for a very long time: that wherever you go, chances are you're recorded in dozens of ways, by dozens of people, and the only actual defense is essentially social ettiquette in handling this content (and the fact that most people just don't care about who you are or what you do).
> "If a complete stranger holds a camera to your face you know what he is doing and you have a legitimate reason to tell him to stop doing it, most of you will probably object to it."
As a street photographer, that last part is entirely false. The vast majority of people have no objections to being photographed in public - almost all reactions I've ever gotten have ranged between "huh? meh" and "oh cool!".
The reactions he got in the video was because he was behaving like Lord Regent of the Universe, not because he was recording someone. My unscientific estimate is that >95% of the people I've ever taken pictures of (all strangers, almost all without seeking permission beforehand), are either ambivalent or downright approving.
Which goes towards the point: Google Glass is the vehicle for the problem, not the problem itself. The problem, as it has always been, is with assholes.