I agree that what should be called "research" should adhere to the highest standards and actively seek to avoid biases.
But generally (not specifically about this psychedelics study), it's OK for us to act upon imperfect, biased information. The absence of a rigorous proof that something works, does not mean it doesn't work. The presence of an imperfect proof is enough to increase the probability of a positive outcome.
A priori you might think there is is a high likelihood that the drug use will just make everything worse.
That is precisely the bias and weakness of this study. Self-medication is almost always a red flag; why not do it in a context of other people that know their stuff, be it tradition, institution (therapy) or even close friends that are mutually invested in each other's long term well-being? DIY approaches to self-transformation are fragile; no one to hold you out of potential pitfalls, to protect you from re-traumatization, to call you out on your self-deception.
The title of the article that it "reduces depressive symptoms" is also telling of this bias. If symptom reduction was equivocal to enduring transformation, weed, cocaine and alcohol would have already healed the entire world. Without the study measuring if the subjects transformed their lives in a way that they were equally symptom free for the next 6 months, 1 year, 5 years, we can't conclude anything curative really took place.
Don't get me wrong, they make a theoretical link to ACT in which experiencing of chronically difficult emotions are facilitated, which does offer potentially huge benefits, but just like we don't use studies on antidepressants that show symptom reduction to go shop bunch of prozac, we shouldn't readily assume we can use this one to facilitate our own healing in a DIY fashion either.
you select a sample of people who want to get better, and are surprised they get better more frequently than a group including people who don't? where's the test on a person depressed and not actively trying to get better taking the same drug? i bet the negative side effects and the death of extra brain cells makes it worse.
yes, if someone tries to use drugs to fix something, you absolutely should stop them. there's a reason you need a doctorate to prescribe those drugs.
new test. my test. I'm going to prove wikipedia makes you a neurosurgeon. let's take 300 highschool seniors who read a wiki article on neurosurgery. let's take another 300 from a random sampling at the dmv. send them all to school for neurosurgery. wow. just wow. the 300 who read the article - a higher percentage of them succeeded in becoming a neurosurgeon. holy crap. wikipedia is amazing.
The selection bias in studies like these is massive. Including only people who actively planned to use psychedelics is a guaranteed way to bias your results toward people who are already true believers. People who voluntarily chose to seek out psychedelics, despite high legal consequences for doing so, are likely already convinced that the psychedelics will "work" for whatever goal they're trying to achieve. Selecting these people over the internet will only further concentrate the group toward people who enjoy discussing and researching this topic online. In short, they optimized the test group for maximum placebo effect.
It's fascinating to watch how internet comments will quickly tear apart any study of prescription medication that has obvious sampling bias, lacks control groups, and relies on self-reported information gathered from internet surveys except when the study tells people what they want to hear.
I'm all for the resurgence of proper, controlled research into psychedelics for treating mental health issues in conjunction with therapy. Let's get to the bottom of this and draw some scientific conclusions on the matter. However, questionable studies like this do more harm than good for the body of research. It may feel good to see studies that claim positive results, but the more we build the body of research on top of obviously flawed studies like this, the harder it is for serious researchers and lawmakers to take it seriously.