There may not be societal costs in the consuming country, but there are certainly negative externalities to the people/countries that produce the coffee beans

I don't know, the guy in Kona running the coffee farm seemed pretty happy. And the coffee was great (and I typically dislike coffee! It was that good...)

In any case, legal production can be regulated in ways that illegal crops cannot. It's hard to draw a legitimate comparison.

I guess I don't consider this a legit argument. You could point to negative externalities that are the result of literally everything humans produce/consume.

But if we want to use that metric, OK. I would venture a guess that the production of coffee yields a heck of a lot less suffering than the production of cocaine, heroin or meth. Any far fewer people consume the latter.

Also to further your point: consider blood diamonds, mining of toxic substances, lead batteries, plastic in the oceans, etc.

Yes! These are all things that rely on the subjugation and abuse of other human and are bad for our earth-spaceship

Alcohol would be a great comparison. Alcohol causes more deaths (and presumably human suffering) than illegal drugs. Only recently has the Opioid crisis caused a dent in these stats.

Even sugar would be a better comparison than coffee.