This experiment was not well-received in its day, but in recent years, the line of thought is becoming popular, as people who work in drug treatment realize that the paradigm is very helpful when treating people in recovery (ie, you have to identify the reason why people feel compelled to seek external relief, be it anorexia/bullemia/marijuana/alcohol/heroin, or else you're just patching over the symptoms).
As for your last point, I'd point out that what most people don't ever see are the people who use drugs responsibly and without any problem - there's a selection bias afoot which causes society to overestimate the harms associated with drugs (and underestimate the harms of the policies associated with them, since they also never see the people locked away).
There are also people who use these as a means of escape or have limited coping resources. It is not that the cigarette is bad, or the pain killer is bad, or sex is bad.
In saying that, having a methadone dispensary four doors down from where I am sitting, I see first hand the obvious effects of drug addiction and how people struggle to escape it's grasp.