See the works of Bruce K. Alexander on this: http://www.brucekalexander.com/
There is a withdrawal in a different sense I think. I had HPPD for around 3 months after heavy psychedelic use with LSD as the centerpiece of that. I was doing up to 10 tabs at once and using mushrooms and certain research chemicals every two or three days for a few months.
Not to excuse my risky behavior, but other people go much harder than I do. MDMA is a favorite choice, especially in Europe where it is cheaper. There are endless stories on Erowid and Bluelight of young people buying 10 grams of a research chemical and plowing through it in a summer. Often it's stuff like MDPV or 25i NBOME, with a short history of human use and carrying cardiac risks.
A wide range of psychedelic drugs have addictive potential IMO. I'm not sure how meaningful the distinction really is between psychological and physiological in the context of addiction.
Compared to these, LSD is much safer. Doses are ridiculously low (most poisons don't work in doses of 100µg), huge overdoses are not immediately dangerous, and an addiction does not form.
One can invent many reasons to ban LSD, but preserving public health can't be sensibly among them.