Some coworkers at my old work quit smoking and also had to give up coffee. The connection was so strong they couldn't have one without the other. And the socializing out back in the smoking area was also a big part of it.
It also can cause friction at work. I've worked with smokers who are practically bouncing with energy. But they were crabby and combative when their nicotine levels were low. Then ten minutes later back to bouncy and cheery, it was exhausting to be around. I also estimated the 10 or 15 minute smoke breaks meant that my coworker made $5/hour more than me when were actually paid the same rate. He was adamant they were not breaks it was just a "quick smoke" but he was gone for 10 or 15 minutes every hour.
That doesn't sound right; the pattern I've generally seen for pack(s)-a-day people is they take similar smoke breaks as pack-a-week people -- something like once every few hours. But they'll smoke multiple cigarettes in that break. This would also make sense if you imagine nicotine decrease rate as constant, but number of cigarettes required to hit "satisfied" as steadily-increasing, though I don't know if that's the case.
Additionally, IME, smoking or not, I've noticed that most people follow the pattern of
1. Do a decent chunk of work for a few hours 2. Dick around for 20 minutes, especially if they just finished something notable
where that 20 minutes get spent on coffee breaks, smoke breaks, chatting around, etc.
Most of the "hard workers" spend more total hours, not more continuous hours, as far as I've seen. Of course, intentional abuse of the break is another matter, but that's independent of smoking
When you leave that context, all you've got left is a costly habit that's a pain to quit.
But, not saying yes at that point might have cost you some chances to socialize and create some bonds with other smokers.
And you can still find places where being a smoker will help you socialize. Ex: in places where smoking is banned in bars, going out for a smoke is a pretty good way to start up a conversation with a perfect stranger.