Personally I don't think there are consequences, but I just feel way less confident and conspicuous when I don't have a drink in my hand. Like I don't belong somehow. I'm sure it's all in my head, but there it is.

I trick I learned long ago at house parties or BBQs is to drink one beer, then constantly re-fill it with water for the rest of the time. That way I have something to hold onto and a way to entertain myself where there is a lull in conversation or whatever.