Try to imagine a Random Controlled Trial protocol for a new depression medication. It is miraculous that any have been approved. It is why you are always seeing articles insisting, idiotically, that RCTs prove depression medications are no better than placebo. Splinting the left shinbone helps only a small fraction of patients who have a fracture.
For validity, RCTs depend utterly on reliable sortative diagnoses. RCT fetishists (which exist, and are common) are mentally unable to process that fact.
You mean Kirsch & Kirsch 1997? Like, that particular study found a really, really small effect in the data submitted to the FDA (so the data chosen by the pharma companies). Even this went away when they used placebos with side effects.
More recent work has suggested that they work for severe depression, which is great but the trouble is that the vast majority of people with a diagnosis of depression don't have MDD (major depressive disorder).
I do agree that mental health is in it's leeches phase right now, and I am deeply concerned for people who end up having to engage with this system.
> It is why you are always seeing articles insisting, idiotically, that RCTs prove depression medications are no better than placebo.
I responded to your claim with a little more colour on that particular trial in order to perhaps start a discussion around what we mean as depression.
Clearly, I didn't succeed at my aim, my apologies ;)
Edit: it's particularly the framing of the trials as "idiotic" that spurred me to reply, so I'd be interested in why you believe this.
> For validity, RCTs depend utterly on reliable sortative diagnoses
So, are you saying you believe an RCT can demonstrate a fact when whichever medication is being tested addresses the actual condition of only a tiny fraction of the participants, and even exacerbates some of the others' various conditions?
Such trials are not idiotic, except as a waste of money. Reporting their results as if meaningful is what I called idiotic.
Or, are you saying you know how to design the protocol for an RCT when you have no idea which, if any, of the participants actually suffer the condition your medication is supposed to treat?
Huh, that's a really good point (which I somehow missed in your original post).
That being said, it's incorrect that it's essential for RCT's to function, they'll still work, but with an attenuation based on the actual treatable population.
> Or, are you saying you know how to design the protocol for an RCT when you have no idea which, if any, of the participants actually suffer the condition your medication is supposed to treat?
Well I don't design said protocols, but I suspect the vaccine manufacturers may be able to point you in the right direction.
I think that your point about reliability of diagnosis is interesting, gonna need to think about it a bunch more, but in general, no diagnosis is without error (as most diagnoses are implemented as checklists, and there's some room for arbitrariness).
Just to note, if your point is correct then it's basically all mental-health, expectancy, placebo, psychotherapy RCT's are essentially worthless, which I suspect is not the right approach to take.
Does it matter if we can diagnose people effectively? Of course.
Does it make it impossible to run an RCT in the absence of effective diagnosis? Not impossible, but it definitely adds an interesting layer of complexity, as well as massively increasing the sample size required.
Consider a pre-trial objective: "We will be testing this medication on patients who have already indicated that it helps with their affliction. Participants will be sorted into groups that get one of (a) a placebo, or (b) reduced, (c) increased, or (d) unchanged dosage relative to the dose they have reported to be effective enough relative to side effects." Such a trial could actually be useful, but good luck getting the epistemology sticklers to sign off on it.
Also, antidepressants characteristically have history-dependent effects. Often, suddenly going off one renders it ineffective if resumed. So good luck with the ethics.
Also: I would like to see the word "affliction" in more medical papers.
I hope you will find gruntling, in time.
In reality mental healthcare feels like only a slight step up from antiquated methods from days long gone. Trying the medication roulette is like spending months at a time throwing darts blindfolded to see if anything helps, hurts or does nothing at all. Waiting months to see professionals who are burned out or just trying their best to not be completely overwhelmed by the demand but are unable to give meaningful care because of how overwhelmed they are. Or not even being able to see a therapist at all because it is cost prohibitive. And then consider basically having to do that on the side while you have your other obligations that you have to tend to. I wouldn't be surprised if only a fraction of people who need help get it, and of that fraction, only a fraction of them see a benefit. It basically boils down to society not caring, or barely caring at most.
In the end I suspect that the efficacy of mental healthcare has its limits, after all it is like trying to fix people after they have been broken, which is way harder than supporting people before they break, but that depends on social structures and support groups that are increasingly weak and strained.