Yes, filming 100% of the time while you are eating at a restaurant would cause unease. Reality TV does this with $10k cameras that are mounted on a dude's shoulder. You don't need Google Glass to know thats a bad idea, so contributing that bad idea to Google Glass isn't a fair comparison because you can do it with other items.
That's weird, and initially I thought glass was cool. It's not.
I had this experience sitting across a restaurant from someone wearing Google glass, and it made me realize what a horrible idea it is.