This argument doesn’t seem sound. Stigma for drugs increases and decreases over time. That wouldn’t happen if it was a question of the drug being inherently harmful or not. You seemed to have missed a third case of opponents of a drug becoming more powerful than its users and sympathizers. See US prohibition of alcohol and the global criminalization of many drugs in ~1971.