News for hackers. Stuff that matters.
Re:AI predicts effective depression treatment based on brain... by piker on Tuesday June 30, 2020 @03:58AM (#23687779)
Is there a bootstrapping problem with these types supervised of approaches?

Per the article:

"Current methods used to diagnose and treat depression are imprecise at best, relying largely on subjective answers to survey questions..."

"... they deployed an algorithm to interpret brainwave patterns unique to individuals with depression..."

Presumably the diagnoses of those "individuals with depression" were based on the same imprecise subjective answers to questions. Thus, is this just modelling the flawed diagnostic?

Re:AI predicts effective depression treatment based on brain... by nabla9 on Tuesday June 30, 2020 @09:02AM (#23689560)
On the contrary, any measurable biomarker for depression is very good news.

In psychology almost all diagnoses are just checklist of symptoms either reported or observed. If the patient checks enough of them, they have that. Accurate knowledge of the underlying causes and conditions or clear tests that verify them don't usually exist.

Depression is extreme version of this, so called diagnostic trash bin. Symptoms of depression are present in many/most other psychological and physical diseases. If the patient shows symptoms of depression first time, it's relatively common for doctor to order full blood panel and other tests to rule out all other causes.

In the end, if nothing else is found and symptoms continue, all that is left is the proverbial "trash bin". If you don't have anything else, but have depression symptoms you are diagnosed with depression. It's not a surprise that medicating depression is just trying different drugs semi-randomly to find out what works.

If we can directly measure for the effectiveness of treatment, it may help to understand how they work, what depression is, or how many different conditions there are behind depression diagnosis.

Re:AI predicts effective depression treatment based on brain... by AuryGlenz on Tuesday June 30, 2020 @01:57PM (#23692836)
I was put in that trash bin for a few years due to always being tired. The only thing I was depressed about was being tired all the time. Eventually I asked for a testosterone panel, which came back really low. That hasn’t completely solved my issues but it would have been enough to put my life on a completely different track.
Re:AI predicts effective depression treatment based on brain... by lvice on Tuesday June 30, 2020 @04:48AM (#23688068)
My understanding is that the diagnoses are eventually right, but they take a lot of trial and error. You would not be modelling the flawed diagnostic if you take the final result of the trial and error process.
Re:AI predicts effective depression treatment based on brain... by visarga on Tuesday June 30, 2020 @08:09AM (#23689236)
Kind of like AlphaZero. Train to cheaply predict the result of a more costly search.
Re:AI predicts effective depression treatment based on brain... by EGreg on Tuesday June 30, 2020 @08:50AM (#23689471)
The question is, what’s the “opponent” here. In other words how do you actually do a trillion rounds of testing?
Re:AI predicts effective depression treatment based on brain... by visarga on Wednesday July 01, 2020 @12:07AM (#23697652)
Unfortunately we are the agents in this game, so it's our bacon on the line in doing the rounds of testing.
Re:AI predicts effective depression treatment based on brain... by 0d9eooo on Tuesday June 30, 2020 @10:42AM (#23690510)
Lots of thoughts.

First, there's a sort of framing issue with "subjective answers to questions" per the press release. These subjective experiences are the very nature of the thing of interest: feeling desperate, crying for help, loss of energy, and so forth. To dismiss them is, at some level, is akin to a chemist dismissing oxygen gas as an appropriate target of study because it's all subatomic particles at some level. These types of press releases often start with this notion, in my mind, because it's necessary to create some sense that the study in question is revolutionary.

I do think they're something of a bootstrapping problem with supervised approaches, which is why in psychopathology and psychiatry research there's a trend toward modeling symptom/behavioral presentations using "unsupervised" approaches. This is partially the impetus behind NIH RDoC, for example, and when you do this, you end up with similar but not identical constructs as DSM diagnoses. The recent DSM (DSM-5) was supposed to address this in part but got mired in politics.

If you look at the actual results, by far the most important features in predicting future symptom state are previous values of those symptoms. So follow-up symptom A is best predicted by baseline symptom A. This autocorrelation importance is a sort of law of behavioral individual differences. The EEG variables are adding in the prediction, but relatively weakly, which further speaks to the central importance of the subjective variables, which is informationally mediating whatever is going on under the skull.

This is really further underscored if you start thinking about things like this: are the EEG signals adding most to prediction of "physical" symptom improvement? There's also a literature about severity of symptoms; it's possible to make predictions of symptom response based on other criteria from previous studies and this paper does little to address that in a head-to-head comparison. K-fold cross validation is also good, but isn't the same as replicating something on entirely different sets of people with different sorts of heterogeneity due to unknown sources.

It's an interesting paper but people need to be cautions about the hype that goes on in these areas (biological psychiatry and AI).