News for hackers. Stuff that matters.
Re:Personality Trait Recognition Using ECG Spectrograms and ... by Calavar on Thursday February 15, 2024 @06:20PM (#39390686)
I'm a junior academic physician with a background in CS whose specific research interest is ECG analysis with deep learning. This paper has some glaring holes.

Apart from the total lack of external validation (which I suspect will never come), we already know that ECGs have features associated with emotion: heart rate, frequency of premature ventricular contractions, frequency of premature atrial contractions. Are these deep learning detectors better than running a simple regression after using something like a wavelet transform to detect these specific features? This matters because if you have a method that is more computationally efficient, more easily interpreted, and more robust to extrapolation to out-of-domain data, why in the world would you go with DL?

Of course the article is completely silent on these questions. If I was a reviewer I would refuse to accept until they added extra experiments to address this.

This is a symptom of a more general problem in interdisciplinary medical research: Papers are written and reviewed either by engineers who have limited understanding of the clinical background or by clinicians who can code a CNN in pytorch but have no knowledge of classic ML methods or ML theory. And so we see seriously flawed papers get accepted. (I know this is a preprint, but I've seen similar stuff in well regarded journals.)

It's incredibly frustrating as someone who tries to do truly rigorous research in this space, only to be desk rejected because ECG magic sounds a lot cooler than incremental gains.

Re:Personality Trait Recognition Using ECG Spectrograms and ... by ramblenode on Thursday February 15, 2024 @07:20PM (#39391283)
> This matters because if you have a method that is more computationally efficient, more easily interpreted, and more robust to extrapolation to out-of-domain data, why in the world would you go with DL?

One reason we will see more deep learning in electrophysiology is simply because it allows one to abstract over a good deal of statistics and signal processing, which are hard. Deep learning is being used as a black box for those who can't or don't want to understand their data.

Re:Personality Trait Recognition Using ECG Spectrograms and ... by Calavar on Thursday February 15, 2024 @08:05PM (#39391710)
I agree with all your points. I think that training algorithms without understanding the underlying data is fundamentally a bad thing. This isn't NLP, where we're willing to put up with LLMs being inscrutable because they blow traditional algos out of the water. Accuracy with deep learning in ECG is moderately better than traditional methods, not paradigm shifting. And unfortunately the drawbacks of DL in ECG analysis are underanalyzed and underdiscussed. Most published DL ECG models that I've played around with have pretty significant failure modes that would preclude them from being used clinically. Of course, it's research, it's not meant to be clinical grade. But the failure modes aren't mentioned at all in the publications, and I've seen very little in the way of research into clamping down on some of these failure modes.
Re:Personality Trait Recognition Using ECG Spectrograms and ... by NeuroCoder on Thursday February 15, 2024 @07:42PM (#39391490)
There are far too few neuro + medicine + CS people. Being able to coordinate research in two of those happens now days we need all three.
Re:Personality Trait Recognition Using ECG Spectrograms and ... by __loam on Thursday February 15, 2024 @08:27PM (#39391888)
Part of the problem is CS people thinking the brain is a computer and biology people thinking ml is magic.
Re:Personality Trait Recognition Using ECG Spectrograms and ... by __loam on Thursday February 15, 2024 @08:25PM (#39391865)
I've done ECG research in the past and these ai reads your mind posts always make me very skeptical.
Re:Personality Trait Recognition Using ECG Spectrograms and ... by 4death4 on Thursday February 15, 2024 @10:37PM (#39392917)
> Are these deep learning detectors better than running a simple regression after using something like a wavelet transform to detect these specific features?

It’s pretty insane to think that a decent neural net trained on sufficient data wouldn’t be able to outperform basic regression.

Re:Personality Trait Recognition Using ECG Spectrograms and ... by ssivark on Thursday February 15, 2024 @11:57PM (#39393336)
What makes you think that a neural net must necessarily be significantly better? How much better is significant? Can one actually train a large enough network on (finite) available data? Sometimes it might take god awful amounts of data+compute+layers to rediscover transforms that we have ready access to, once math has matured enough.
Re:Personality Trait Recognition Using ECG Spectrograms and ... by 4death4 on Friday February 16, 2024 @09:45AM (#39397699)
It’s logically obvious. You can recreate any linear regression using a neural network. So a neural network approach will always be at least as good as linear regression. But a neural network can model non-linear relationships as well. Now ask yourself, what is the likelihood of there being a non-linear relationship in the data? As with most real-world data, the likelihood of non-linear relationships is extremely high. That’s why we can be quite certain a neural network approach will able to outperform linear regression.
Re:Personality Trait Recognition Using ECG Spectrograms and ... by Calavar on Sunday February 18, 2024 @09:55PM (#39425811)
> It’s logically obvious. You can recreate any linear regression using a neural network. So a neural network approach will always be at least as good as linear regression.

Well, maybe it's logically obvious, but it's also empirically false: https://www.youtube.com/watch?v=x7psGHgatGM (Skip to 13:36 to get to the point)

> But a neural network can model non-linear relationships as well. Now ask yourself, what is the likelihood of there being a non-linear relationship in the data? As with most real-world data, the likelihood of non-linear relationships is extremely high. That’s why we can be quite certain a neural network approach will able to outperform linear regression.

As I've said elsewhere in this thread, a deep neural network of N layers can be decomposed into a complex nonlinear function of N - 1 layers followed by a linear combination of the nonlinear features generated that function. There's no law that says that you have to use a neural network to generate your nonlinear features. You can use any method you like and then linearly combine those.

Re:Personality Trait Recognition Using ECG Spectrograms and ... by 4death4 on Monday February 19, 2024 @03:57PM (#39434713)
I don’t think you understand what you’re saying. “A linear combination of non-linear features” is not what people mean when they talk about linear regression. And even if they were, a neural network will do a much better job of generating the non-linear features than you will be hand. So again, it’s logically obvious that a neural network will be capable of performing better than simple linear regression.
Re:Personality Trait Recognition Using ECG Spectrograms and ... by Calavar on Sunday March 03, 2024 @07:24PM (#39585813)
> I don’t think you understand what you’re saying. “A linear combination of non-linear features” is not what people mean when they talk about linear regression.

According to who? Did you watch the talk I linked? Because in that video, NeurIPS gave their test of time award to a paper where they generated nonlinear features and used those to train a linear classifier. I guess the committee at NeurIPS also didn't understand what they are saying?

> And even if they were, a neural network will do a much better job of generating the non-linear features than you will be hand. So again, it’s logically obvious that a neural network will be capable of performing better than simple linear regression.

Again, logically obvious but often empirically false.

Re:Personality Trait Recognition Using ECG Spectrograms and ... by Calavar on Friday February 16, 2024 @12:47AM (#39393629)
It's really not. If you throw a blob of unstructured data at a neural network, then sure, it will beat everything else by a mile. If you actually try to understand your data and engineer your features, you can often beat DL with much simpler methods. (Obviously YMMV depending on the specific problem domain.) Unfortunately, feature detection and feature selection are dying arts. Just throw all the data at a CNN or transformer, report the accuracy, and that's it, that's the paper. No analysis of robustness, edge cases, etc. because you can't easily do those things if you haven't taken the time to understand your data in the first place.
Re:Personality Trait Recognition Using ECG Spectrograms and ... by seertaak on Friday February 16, 2024 @03:49AM (#39394687)
There's definitely xor type nonlinearities going on in factor models of personality types. The interaction between two factors is often very different at the extrema of each factor - often flipping sign.

Regression models can't capture that. Now it's possible that the behavioral nonlinearities are somehow emergent, and that despite those, the ECG can be captured by a linear model. But I wouldn't want to bet on that.

Re:Personality Trait Recognition Using ECG Spectrograms and ... by Calavar on Friday February 16, 2024 @04:14AM (#39394852)
Nonlinearities in ML existed long before DL and ReLU. What's at the end of a deep CNN? Probably global pooling followed by a dense layer? That final dense layer is taking a weighted combination of the last stack of feature maps. So deep learning is fundamentally the same as the old approach: Construct features with a nonlinear transformation of the input. Then calculate a score by taking a linear combination of those features. The only difference with deep learning is that the manner of constructing those nonlinear features is left unspecified.
Re:Personality Trait Recognition Using ECG Spectrograms and ... by seertaak on Friday February 16, 2024 @05:13AM (#39395203)
Interesting comment. I'm not an expert in CNNs but what you're saying makes a lot of sense.

Question: are you saying the final layer is "in effect" a linear combination? At least in transformer architectures, the dense end block is iirc three layers deep and uses relu. Even if the CNN's dense final part is one layer deep, wouldn't it also be using relu activations?

Even a one layer dense ANN can capture nonlinearities if it has a nonlinear activation fn. But maybe in practice the activations don't do a lot of work? Or am I simply mistaken about final layer, does it simply have linear activations?

Also, can you share your intuition about the nonlinear features? Spectral/wavelet analysis? Or something more complex?

Re:Personality Trait Recognition Using ECG Spectrograms and ... by Calavar on Sunday February 18, 2024 @07:27PM (#39424976)
> Question: are you saying the final layer is "in effect" a linear combination?

Not just in effect a linear combination; it is a linear combination. There are some exotic nonlinear NN layers that are used in particular niches. But in general, NN layers are syntax sugar over matrix multiplications (i.e. linear functions). The nonlinearities are the activation functions between the layers.

> But maybe in practice the activations don't do a lot of work?

No, the activations do do a lot of work. There are real nonlinearities in the network.

> Or am I simply mistaken about final layer, does it simply have linear activations?

Sometimes you actually do use a linear activation on the final layer for certain regression tasks, but that's not the main thing I'm getting at.

Let's say that your final layer has is a ReLU activation. What is this conceptually? You are taking a linear combination of the features from the previous layer and them clamping the result to >= 0. Sure, that's a nonlinearity, but it isn't going to have much in the way of emergent modeling capabilities. You need to stack many, many nonlinearities before you get that.

So my point is that a deep neural net of N layers boils down to a complex nonlinear function of N - 1 layers, followed by a "dumb" linear combination in the final layer. You can do this with traditional ML methods as well, but you have to handcraft your nonlinearities.

> Also, can you share your intuition about the nonlinear features? Spectral/wavelet analysis? Or something more complex?

There's innumerable possibilities here. It could be starting with a method that's inherently nonlinear, like nonlinear PCA, polynomial regression. Or it can involve transforming the output of a linear function (like the Fourier transform) in a nonlinear way.

Admittedly this very tough. And for really tough problems, like video synthesis, effectively impossible. But NNs get thrown at much simpler problems all the time.

Re:Personality Trait Recognition Using ECG Spectrograms and ... by refibrillator on Friday February 16, 2024 @01:12AM (#39393772)
If regression on wavelet coefficients solves your task then sure DL is unnecessary.

I spent half a decade on a data science team developing algorithms for ambulatory ECG monitoring at scale. 100+ human years of 256 Hz multichannel ECG collected everyday. Waves, beats, & rhythms all segmented and classified. PDF reports generated for cardiologists and EPs.

DL techniques like unsupervised pre-training blew away every other classical or purely supervised method we ever tried. Especially for rare and critical arrhythmias like VF. It unlocked value in massive unlabeled datasets. It gave us a framework for scaling model performance as a function of dataset size and compute, with no fundamental limit.

I do get your frustration about the publication game. You might find industry is more aligned with pragmatism & efficiency.