That was my experience as well. In my later teenage years there was a pretty successful movie around where the protagonist develops schizophrenia after using LSD one single time and his subsequential descent into "madness". That scared me off psychedelics for decades.