That's one of the reasons I'm looking forward to AirPods. I can use Siri with the earbuds I have, but their microphones are crummy enough to require I do so somewhat loudly. While I don't always mind seeming like I'm talking to myself in public, and indeed there's the odd occasion when seeming unpredictable and potentially dangerous can be positively beneficial, having the option of greater discretion would be nice as well.
Also, FWIW, I've gotten pretty good at discreetly talking to my wrist in public. In any moderately loud area I don't think anyone even notices me doing it.