>With SAINT, study participants underwent 10 sessions per day of 10-minute treatments, with 50-minute breaks in between. After a day of therapy, Lehman’s mood score indicated she was no longer depressed; it took up to five days for other participants. On average, three days of the therapy were enough for participants to have relief from depression.
One imagines that three to five 10 hour (!) days of a presumably highly-trained professional spending time and attention and focus on a patient serves as no less than an incredibly strong placebo. It would be interesting to see a blinded study where the transducer functionality was replaced by a small heater (or whatever is needed to make the system act, feel and sound the same for all involved).
It would be lovely if this held up. Depression and other psychological issues are insidious.
Talking to a therapist 1 hour a week doesn't help you manage those waves of depression you feel 20x a day 6 other days of the week. If you are going to change any subconscious or neurological patterns, you need to intervene in those moments of symptoms.
Either that preexisting intervention also would have a big success rate, in which case this isn’t news and just isn’t applicable widely due to scale, or else this effect is not due to frequent attention / intervention.
> The researchers are conducting a larger, double-blinded trial in which half the participants are receiving fake treatment.
I'd think I'd either figure out that the person giving me therapy was talking nonsense pretty quickly, or their 'placebo' therapy is indistinguishable from actual therapy, in which case am I still a control?
IMO the only way to blind this is to put people in an automated machine that either turns the magnet on or it doesn't (and it makes some noises either way). You can't have highly trained professionals in the loop making people feel valuable by giving them all this attention, as that could be a confounding effect in both the alleged-treatment and the alleged-placebo.
It's fine as long as both the experimental group and the control group get the same amount of attention, right? If both groups show the same amount of improvement, that means it was caused entirely by the attention, not by the magnetic treatment.
(I agree that this is what the "double" in "double blind" means, but that's not what we were talking about.)
Note that with Reiki, it's a little more complicated to properly blind (this is an issue for pretty much every physical therapy, and surgery).
It should still be distinguished from non-placebo effects where possible and I agree with all of the concerns and worry expressed in this thread.
> Those with the high-COMT variant had the weakest placebo responses, and those with the opposite variant had the strongest. These effects were compounded by the amount of interaction each patient got: For instance, low-COMT, high-interaction patients fared best of all, but the low-COMT subjects who were placed in the no-treatment group did worse than the other genotypes in that group
> Analyzing the data amassed during the first 10 years of the study, Hall found that the women with the low-COMT gene variant had significantly higher rates of heart disease than women with the high-COMT variant, and that the risk was reduced for those low-COMT women who received the active treatments but not in those given placebos. Among high-COMT people, the results were the inverse: Women taking placebos had the lowest rates of disease; people in the treatment arms had an increased risk.
It means you don't have to buy the expensive magnet. You just get people to hang out with a patient for a couple of days and they'll respond better than 10x the average response of an SSRI.
To get a feel for the impact, an SSRI makes you an inch taller, the placebo effect makes you another inch taller. And this intervention would make you 10 inches taller.
Very very annoying that the press release over-hypes this.
Maybe she was lonely and all that time with technician was enough to help her perk up :)
Lots of research in CBT is plagued with abuse of statistics, manipulation of randomized controlled trials, sensalization, and over-hype. For the vast majority of time CBT is good for symptomp reduction and patients relapse frequently. Frankly, the depressed people I know in real life, I don't know how many of them would be benefitted by CBT style therapy.
My own experience is it took me almost a year of therapy, reading psychology and philosophy, lots of self-relfection and going to dark places in my mind to see marked improvement in my depression. If you are suffering from depression, please give psychotherapy a try. It'd be hard, but it's worth it!
Concerning the approach, this study looked into a fairly new form of rTMS, namely iTBS. Here is paper detailing how it differs from 10Hz rTMS, with a specific focus on potential cost savings: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6742475/
We certainly need more data on iTBS effectiveness and I hope that a properly controlled, double-blind study can yield some information on whether this is actually effective on a larger scale and in the long term. A sample size of 21 is simply to small to make any judgement on that.
Also I don't think your analysis that it's all or nothing, i.e. that it has a true effect size of "Nobel prize winning" or nothing is true. I think it's very likely that there is some effect size it's just not world changing.
Huge RCTs with many thousands of subjects only sometimes find effects for SSRIs which are the first thing doctors prescribe for depression after telling you to go get some exercise and sleep better. This study has N=21, isn't an RCT, and isn't even claiming to be studying efficacy. In terms of result robustness, this is about as strong as posting a poll in your team Slack channel.
Anyway, the study wasn't even focused on efficacy! It says so in the paper. But that didn't stop Stanford PR from putting the supposed efficacy in the headline anyway to get clicks/shares.
This has everything to do with small effect size for SSRIs and nothing to do with this study. Doctor's prescribe SSRI's not because they are especially effective but because they are especially safe. For example MAOI's have much higher effect sizes than SSRI's. Also I'm pretty sure every single RCTs with thousands of subjects regarding SSRI efficacy has found an effect, just not always a large one, but I'd love to see a study as a counter example.
But on this to this study
What do you think a control group remission rate would be 20%, with ECT it's 48%. They had a remission rate of 90%. An effect that size is most likely a true effect or outright fraud. What is the probability this happened by accident? My probability is rusty but that seems like that would be highly unlikely. It's gotta be less than 1/1000.
If you gave a pill you thought might help people lose weight to 21 people and 90% of them last 80 lbs. Would you think "hey there's no control group", or "holy shit I probably found something".
That's just math.
The question is how believable it is that the entire effect is attributable to the magnet intervention.
Exciting research results announced in university press releases turn about to be something cool infrequently to put it mildly (also true for small studies, also true for medical results, also true for self-reported studies, etc), so my prior is that most or all of the effect will go away.
This study showed a 30 point improvement on the MADRS, that's from 35 -> 5. That's insane, the typical anti-depressant move a patient 3 points.
What happened? 21 people got this technique applied to them, and 19 of them achieved remission. The average person in this study went from severely depressed to not depressed in 5 days. I'm a big fan of Gelman and the garden of forking paths, but you can't fork your way to a result like that.
Something interesting happened here, even if it's not the therapy.
It's just so frustrating that professional scientists just completely ignore this statistical fact. Why not use use N >= 100 or not at all. It just confuses things in my opinion.
There are multiple reasons why small N is not bad thing.
Firstly, multi year research projects are not silent for years or decades before they deliver final paper. They publish interim research, technical reports, exploratory research or preliminary research to clarify issues and fine tune the direction of research. You do something and you publish it.
Secondly, multiple studies where N is small can be more reliable than one big study with large N. There are errors other than statistical errors.
Thirdly. Effect size is much more important than N. N=1 can win you a Nobel price if the effect size is big enough. Grow a mice that weights 10 kg and N being 1 or very small number is good enough. There are many studies where N is tens of thousands (p < 0.001) but the effect size is so small that the result is not scientifically interesting.
Finally, statistics is not the final arbiter of importance of the study. It has become standard to add it into every publication, no matter how small. But researchers don't focus on that.
---
Everyone wants to participate in the discussion and most people in comment sections know something about statistics but little to nothing about the subject, so you always get the 'correlation is not causation', 'small N' so it's irrelevant talking points.
Generally because bigger N means the study is more complicated logistically ( cost, finding participants, whatever). And if the results are promising you can always get more funding to move on to bigger trials. Nobody is going to pay for you to recruit 1000 participants if you can't show an effect size with a smaller number first.
> I think its pretty obvious that a small N is not statistically significant
Well this totally depends on the effect size. You can't just dismiss results because N is small, that ignores how probability works.
>Why not use use N >= 100 or not at all.
Because N > 100 is a totally arbitrary value. The size of N that you need is totally dependent on the effect size, not an arbitrary value. You'd miss lots of real but small effects by having an N of only 100. You'd miss out on lots of real but large effects which can be observed at N < 100.
Basically this approach would severely limit the branches of research we can do, for a questionable gain in statistical reliability.
This seems to be such a case. I wonder whether it was set from the beginning with more funding in mind.
Say you're a tribal chieftain. One of your men (one among thousands) finds a berry bush. Another eats a berry from the bush. He promptly dies. You're a scientific tribe, though, and you know N must be high before you conclude the berries are poisonous. So, one of the other guys eats another berry. He, too, dies immediately. The process repeats until it's you considering the bush. How does the situation change if the number of people before you was 19 vs 21?
I agree with nabla's comment that the real problem is our inability as laypeople to properly understand (or even properly upvote) scientific work.
What's unfortunate is that the research gets published at this stage beyond the scientific community that understands how inconclusive it is. Because in the long run that just contributes to the erosion of public trust in the scientific method.
These are preliminary studies: someone says, "Let me try something," it appears to work, and they report what they saw, with the resources they had.
As a result, someone organizes follow-on studies with more resources, which costs much more. At that point, the effect usually disappears, due to the greater power of the more careful study.
Most people have 2,4,5,6,7.
The criteria is so vague your study would have to control for each item on the list. Here, they're not even using that criteria, and instead just using a simple test which involves self-reports. Placebo could be a huge factor here.
[1] https://www.psycom.net/depression-definition-dsm-5-diagnosti...
"To receive a diagnosis of depression, these symptoms must cause the individual clinically significant distress or impairment in social, occupational, or other important areas of functioning."
You might accuse this definition of being too subjective. But it is certainly not something that includes "almost everyone".
I'd say most of the lower class has this issue.
> But it is certainly not something that includes "almost everyone".
How do you know this?
edit: maybe almost everyone is too much, I mean about 97% of the population.
Why would you say that? Do you think that rich people are happy and poor people are depressed? Anecdotally, the poorest people I know are often the happiest and the richest people are often the most miserable. I was a whole lot happier when I was younger and poorer. One thing has nothing to do with the other.
There are numerous studies showing this, for almost all disease categories. All psychological illness is negatively corelated with socioeconomic status.
https://en.wikipedia.org/wiki/Socioeconomic_status
I'd say it's hard to find good sources of nutrition and the time to exercise when you're part of the lower class.
edit: negatively correlated...
Also depression. It took me one minute to find three sources, could you please make some effort here? I have other things to do.
And you're right, it's not a causative factor necessarily, but I think there's pretty strong evidence at this point, it's hard to find a poor person who doesn't have some condition.
[1] https://www.sciencedirect.com/science/article/abs/pii/S00223...
[2] https://psycnet.apa.org/doiLanding?doi=10.1037/0021-843X.111...
[3] https://ajph.aphapublications.org/doi/full/10.2105/AJPH.93.1...
If most people had this symptom, you wouldn't see the current movie, gaming, and music sales. Also, sex work would not be a lucrative job, and neither would the leisure industry (except perhaps casinos).
> A slowing down of thought and a reduction of physical movement (observable by others, not merely subjective feelings of restlessness or being slowed down).
If an overwhelming majority of people had this symptom, the world would have crawled to a stop by now. One look out the window on a busy street shows that most people are walking around normally, often at a brisk pace.
> Fatigue or loss of energy nearly every day. > Feelings of worthlessness or excessive or inappropriate guilt nearly every day.
These ones I could grant are extremely common, especially among the working class, and are often encouraged by moder work and management practices.
> Diminished ability to think or concentrate, or indecisiveness, nearly every day.
If this were true of the vast majority of the population, you wouldn't have any kind of political life outside the elites, you wouldn't have any real amount of productive work, and you wouldn't see any art or creativity from the working class, which is obviously not true.
Overall your claim that ~97% of the population suffers from all of these symptoms is simply obviously false.
> I'd say most of the lower class has this issue.
The lower class is not "in significant distress or impairment in social, occupational, or other important areas of functioning because they have depression-like symptoms. It is the other way around: a lot of them have horrible jobs where they are treated poorly, and that is causing depression-like symptoms (especially fatigue and feelings of guilt). It is not the fatigue that keeps them from living a better life, it is the life they are forced to live by our current economic and societal model that is causing their fatigue.
Depression manifests very differently, especially if you are close enough to the depressed person to get a good glimpse into their life. Diminished pleasure or interest does not manifest by being slightly bored while watching TV or listening to soul-less pop. It manifests by sitting alone in the dark, sleep-less, for hours on end. It manifests by simply not feeling powerful enough to get out of bed, regardless of how hungry you are, or of how important your job is for your livelihood. It manifests by crippling guilt and long bouts of crying, regardless of how important for your future it is to greet customers with a smile at Walmart. People with clinical depression rarely hold down jobs, especially if they are untreated. They rarely hold down relationships of any kind.
The symptoms described above sound vague, but they are actually very recognizable. Sure, you could say that 97% of the population has something that vaguely looks like these symptoms. But remember that this is a guide for trained psychiatrists, it is not meant for the general public to diagnose others.
Perhaps they're in the same category? All psychological disease categories are co-morbid with each other.
What about existential crises? Do people who have those also have depression?
Not to say that people don't incorrectly make diagnoses all the time.
Many of these studies also rely on rating scales (typically HAMD or even PHQ9 for depression) that are a measure of depressive symptoms but don't in themselves have the power to make a diagnosis.
This is my point again if you read my original post, thank you. :)
I'd go further and say the DSM-V is pretty bad at narrowing down which medication to use for depression. I know so many people on damn SNRIs whose diet is complete trash.
(Caveat: my wife is a psychiatrist. We wrote a book on this: https://mhnav.com/book/)
A few reasons I don't think it will hold up: - this is a university press release - N = 21 - no control group - no blinding - university press release doesn't even claim it's a new approach, just incremental improvement on an existing approach - the paper (link below) is about evaluating safety, feasibility, and "preliminary efficacy" (i.e., they're not advertising this as a robust result)
Paper available here:
https://ajp.psychiatryonline.org/doi/10.1176/appi.ajp.2019.1...
https://www.biorxiv.org/content/10.1101/581280v3.full