I don't remember ever having seen _any_ security guards in a grocery store before, let alone armed ones! It's possible I've mistaken some for regular employees if they wore the same uniform, but I'm fairly certain I would have noticed if they were carrying weapons...
I don't think their purpose is to deter shoplifters; it's really to deter assaults on staff. Shoplifting is just part of the cost of doing business as a grocers.
I once saw a couple of armed cops in my local grocers; UK cops are generally not armed, but these guys were carrying big old pistols as they wandered through the aisles looking for sandwiches. I'm not keen on sharing supermarket aisles with gunmen, even if they're in uniform; so I complained. Armed cops should disarm (e.g. leave their guns in the cop-car) if all they're doing is shopping for lunch. I was treated with scorn.
I rather politely told them to fuck off. (We can do without militarisation of our police service thankyou.)
Larger stores in the suburbs definitely do, and my understanding is something like whole foods would be similarly sized to those.