Personality trait recognition using ECG spectrograms and deep learning

by PaulHoule • 2 years ago

48 40 2 years ago

Very nice. Now let's see the external validation.

PaulHoule2• 2 years ago

Would be nice to see. Doesn't look like a hard experiment. Even a 12-lead ECG is pretty easy to set up compared to an EEG, PET scanner, or something like that. There's a very direct line from the heart to the sympathetic and parasympathetic nervous systems so it's quite believable that at least some of the "Big 5" should manifest in HRV. The reliability of their test doesn't seem far off from conventional personality tests

https://www.psychologytoday.com/us/blog/people-unexplained/2...

equanimitivity• 2 years ago

The ASCERTAIN dataset comprises a diverse range of physiological signals, including ECG recordings, collected from 58 participants exposed to video stimuli (36 videos) categorized in different categories based on valence and arousal levels. In particular, there are four subcategories of these 36 video clips. Clip 1 to 9 is categorized into High Arousal and High Valance (HAHV), Clip 10 to 18 Low Arousal and High Valance (LAHV), Clips 19 to 27 Low Arousal and Low Valance (LALV), and 28 to 36 High Arousal and Low Valance (HALV) clips [10]. ECG signals from the right and left arm were recorded at a sampling rate of 256Hz… our model outperformed the closest rival by a wide margin (0.56), achieving an accuracy of 0.94 for the extraversion trait. Similar trends are seen for agreeableness (0.92 versus 0.55), conscientiousness (0.92 versus 0.60), emotional stability (0.93 versus 0.53), and openness (0.93 versus 0.48).

Predicts Big 5 based on heart beat response to various stimuli. I wonder if EKG’s really contain more information beyond heart rate as the conduction within the heart is pretty consistent normally, regardless of rate. Neural nets seem to work great at finding any signal- too bad it isn’t so clear what those signals are.

PaulHoule2• 2 years ago

I can't tell if they are using a 2-lead or 3-lead ECG. Personally I think it's pretty cool that a 12-lead EKG can see the electric field generated by your heart as a vector

https://en.wikipedia.org/wiki/Electrocardiography

I've had health care professionals wire me up for a 12-lead ECG (always normal so far) in less than a minute too.

FrustratedMonky4• 2 years ago

Know it's a loose analogy.

But we really don't know how AI works, really, when we peer into the memory.

And earlier today someone posted how to Visualize internals of Mistral 7B.

And we don't really know how brain neural net works.

And here is a study peering into it.

And in both posts, the visual output is very similar.

They are both neural nets really.

https://news.ycombinator.com/item?id=39378773 https://github.com/valine/NeuralFlow

__loam5• 2 years ago

I really hate when people post idealogical nonsense like this when looking at cross-displinary work. It almost feels like a religious level of arrogance to make these comparisons.

There are some interpretability problems with modern machine learning models but we have perfect information about how they work. You can observe their state directly and the math backing them is very well understood. Contrast that with the brain where observability is extremely limited and how it works on the tissue scale still isn't that well understood.

The similarities are surface level and, imo, a distraction to making serious progress in both fields. Talking about how these things mimic the brain is more marketing than science.

FrustratedMonky4• 2 years ago

1. Where was there any reference to any 'ideology'?

2. Cross Silo ("disciplinary") work is often misunderstood, rebelled against.

3. "Perfect Information" isn't understanding. "Observing State" isn't understanding. See the dozens of articles on AlphaGo.

4. Yes, there is still a lot to learn about the brain. That doesn't mean we don't already know a lot. There are neurons right? There are electrical potentials right? It exists in the physical universe right? It can be modeled right? So when do the models scale up enough to match reality? You can't say 'Never'.

livinginfear• 2 years ago

Can anyone predict a potential benefit to society that might come from this research? I can only think of bad things coming from this. It's like a major step down the staircase towards dystopia.

Everyone was up in arms the other day about a developer who had to take a personality test for a job at FedEx. Imagine what this leads to.

boredemployee• 2 years ago

I can't wait for this technology to be used by HR. /s

andy99• 2 years ago

Phrenology with a few modern buzzwords thrown in.

debbiedowner1• 2 years ago

Did I miss something or did they not hold back any validation data? The whole thing would be hopelessly overfit, correct?

belval• 2 years ago

They use 10-fold cross-validation, but yes imo this entire paper does not pass the smell test. They used pre-trained Resnet and ViT without specifying pre-training on what so Ièm 99% sure that they picked their weights from the pytorch model zoo or similar and as applying an ImageNet-pretraining to ECG classification.

Calavar7• 2 years ago

I'm a junior academic physician with a background in CS whose specific research interest is ECG analysis with deep learning. This paper has some glaring holes.

Apart from the total lack of external validation (which I suspect will never come), we already know that ECGs have features associated with emotion: heart rate, frequency of premature ventricular contractions, frequency of premature atrial contractions. Are these deep learning detectors better than running a simple regression after using something like a wavelet transform to detect these specific features? This matters because if you have a method that is more computationally efficient, more easily interpreted, and more robust to extrapolation to out-of-domain data, why in the world would you go with DL?

Of course the article is completely silent on these questions. If I was a reviewer I would refuse to accept until they added extra experiments to address this.

This is a symptom of a more general problem in interdisciplinary medical research: Papers are written and reviewed either by engineers who have limited understanding of the clinical background or by clinicians who can code a CNN in pytorch but have no knowledge of classic ML methods or ML theory. And so we see seriously flawed papers get accepted. (I know this is a preprint, but I've seen similar stuff in well regarded journals.)

It's incredibly frustrating as someone who tries to do truly rigorous research in this space, only to be desk rejected because ECG magic sounds a lot cooler than incremental gains.

ramblenode• 2 years ago

This matters because if you have a method that is more computationally efficient, more easily interpreted, and more robust to extrapolation to out-of-domain data, why in the world would you go with DL?

One reason we will see more deep learning in electrophysiology is simply because it allows one to abstract over a good deal of statistics and signal processing, which are hard. Deep learning is being used as a black box for those who can't or don't want to understand their data.

Calavar7• 2 years ago

I agree with all your points. I think that training algorithms without understanding the underlying data is fundamentally a bad thing. This isn't NLP, where we're willing to put up with LLMs being inscrutable because they blow traditional algos out of the water. Accuracy with deep learning in ECG is moderately better than traditional methods, not paradigm shifting. And unfortunately the drawbacks of DL in ECG analysis are underanalyzed and underdiscussed. Most published DL ECG models that I've played around with have pretty significant failure modes that would preclude them from being used clinically. Of course, it's research, it's not meant to be clinical grade. But the failure modes aren't mentioned at all in the publications, and I've seen very little in the way of research into clamping down on some of these failure modes.

NeuroCoder• 2 years ago

There are far too few neuro + medicine + CS people. Being able to coordinate research in two of those happens now days we need all three.

__loam5• 2 years ago

Part of the problem is CS people thinking the brain is a computer and biology people thinking ml is magic.

__loam5• 2 years ago

I've done ECG research in the past and these ai reads your mind posts always make me very skeptical.

4death43• 2 years ago

Are these deep learning detectors better than running a simple regression after using something like a wavelet transform to detect these specific features?

It’s pretty insane to think that a decent neural net trained on sufficient data wouldn’t be able to outperform basic regression.

ssivark• 2 years ago

What makes you think that a neural net must necessarily be significantly better? How much better is significant? Can one actually train a large enough network on (finite) available data? Sometimes it might take god awful amounts of data+compute+layers to rediscover transforms that we have ready access to, once math has matured enough.

Calavar7• 2 years ago

It's really not. If you throw a blob of unstructured data at a neural network, then sure, it will beat everything else by a mile. If you actually try to understand your data and engineer your features, you can often beat DL with much simpler methods. (Obviously YMMV depending on the specific problem domain.) Unfortunately, feature detection and feature selection are dying arts. Just throw all the data at a CNN or transformer, report the accuracy, and that's it, that's the paper. No analysis of robustness, edge cases, etc. because you can't easily do those things if you haven't taken the time to understand your data in the first place.

refibrillator• 2 years ago

If regression on wavelet coefficients solves your task then sure DL is unnecessary.

I spent half a decade on a data science team developing algorithms for ambulatory ECG monitoring at scale. 100+ human years of 256 Hz multichannel ECG collected everyday. Waves, beats, & rhythms all segmented and classified. PDF reports generated for cardiologists and EPs.

DL techniques like unsupervised pre-training blew away every other classical or purely supervised method we ever tried. Especially for rare and critical arrhythmias like VF. It unlocked value in massive unlabeled datasets. It gave us a framework for scaling model performance as a function of dataset size and compute, with no fundamental limit.

I do get your frustration about the publication game. You might find industry is more aligned with pragmatism & efficiency.

exabrial• 2 years ago

I can't wait for my next job interview where they give me an EKG.

e12e• 2 years ago

Hah, the future is the other way around - company runs EKGs on teams and extract a vector per team, pick vectors for teams with best stack rating or some other gamed-to-death "productivity metric".

Hand vector to recruiting agency and have them run EKGs on the desperate masses. Hire copies of your best team. Profit.

Season with other "data points" like DNA tests.

tudorw• 2 years ago

"Whenever the person makes an error, a special pattern of brain activity shows up: a sharp, negative electrical activity that is strongest at the top of the head. Since this electrical activity is negatively charged and associated with making errors, it is called the error-related negativity or ERN'..."A curious thing about the ERN is how quickly it happens after you make an error. So quickly, in fact, that it happens before you are aware of your mistake. The ERN usually occurs no later than 100 ms (1/1,000 of a second) after an error has been made. " https://kids.frontiersin.org/articles/10.3389/frym.2020.0008...

So if I wear one of these, I can get a desklamp that flashes when I make a mistake, it's going to be like a strobe :)

antihipocrat• 2 years ago

The researchers are arguing that using ECG reduces subjectivity in personality assessment. Which is predominantly determined by self reported questionnaires.

They are mapping ECG data to the 'big 5' personality traits, and neglected to mention how these traits were determined for each participant prior to measuring the model's predictive accuracy. Almost certainly these primary trait scores were determined through a questionnaire.

Does this have any more validity than the E-meter from Scientology?

From the paper:

Traditionally, assessment for personality traits has relied heavily on self-reported questionnaires and behavioral observations. While these methods have proven valuable, they also introduce subjectivity and potential biases

biosboiii• 2 years ago

This is the new shit.

Measure some kind of data which most people don't fully understand (e.g. ECG spectrogram), label it with arbritrary bullshit (skin color, political beliefs, COVID positive/negative), apply some kind of ML/LSTM black magic, add some data, subtract some training data, so you get a more or less good fit.

Publish the paper, bam! data science/machine learning!

beryilma• 2 years ago

Using the spectrum of the ECG is a non-starter unless the ECGs fed to the DL algorithm (after the model is trained) for detecting the personality traits are obtained under the same conditions of the original datasets (while watching a short video, sitting on a chair, etc.)

The spectral content of the ECGs will change depending on the heart rate. So if you run the DL model using an ECG collected while the subject was, say, running will generate completely bogus results.

Personality trait recognition using ECG spectrograms and deep learning

Related Stories