This varies with the drug. MDMA is notorious for losing its "magic" after six or so trips. There's a special empathic, soul-opening quality to it - the kind of thing that makes it such a good candidate for treating PTSD. But after a few times, it becomes more like a regular psychedelic amphetamine.. fun, but not life-changing.

On the other hand, pure psychedelics don't really lose their magic. Tolerance develops, but goes away after a week. Psychedelics aren't associated with brain damage, while MDMA mildly is.

Then there's opioids, of course, which are ruinously addictive and fickle with tolerance.

When you're on the outside of the community, it's easy to lump "drugs" together and make generalizations. But every class of drug is different.