Study finds Lumosity users show no improvements in cognitive performance

https://arstechnica.co.uk/science/2017/07/lumosity-doesnt-improve-brain-function/

by sjcsjc • 9 years ago

156 75 9 years ago

arstechnica.co.uk

moron4hire• 9 years ago

I'm in the middle of planning a startup with a doctor who does research on therapy methods for traumatic brain injury. She frequently goes on rants about how much bullshit Lumosity is. So literal appeal to authority, but this particular authority was doing research at NIH for 17 years. I think she knows what she's doing.

notheguyouthink2• 9 years ago

Are there any that aren't bullshit?

I'm debating just setting up a spaced repetition app with my own custom knowledge base. At least that way I may not be getting higher brain function, but higher retention on memory items I care about.

dmreedy2• 9 years ago

If there are, they would be as difficult to recognize as germ theory to a 16th century doctor. We simply don't know enough yet about the mechanism.

DiffEq• 9 years ago

Or...one, or all of these:

1.Learn a Second Language 2. https://projecteuler.net/archives 3. http://rosalind.info/problems/list-view/

If you are going to exercise your brain, you might as well get smarter and learn useful skills in the process.

aklemm• 9 years ago

Why not just build stuff and solve real life problems with all your effort? That in itself is brain training AND productive at the same time.

kiba• 9 years ago

Spaced repetition absolutely works. I can literally see it in the statistics as clear as day. This is in addition to all the scientific evidences.

gargarplex2• 9 years ago

I'm conversational in Swedish. Started with duolingo.

Bartweiss7• 9 years ago

...basically no.

Dual n-back was thought to be the only thing that worked, but even that now looks non-transferrable. (Gwern is great on this topic.) For young, healthy adults, I do not know of any brain exercise capable of directly raising IQ or its analogues.

You might, maybe, be able to boost neurogenesis. Exercise is a decent candidate for that, and I don't know the situation with drugs there. But I don't know of games that do it.

Realistically, you've got a few options. Your idea, spaced repetition of useful stuff, is great. Practicing memory tricks like mind palaces can improve your long-term memory on arbitrary facts. Picking up new topics dissimilar to the ones you know probably helps synthesis. Lots of things can maybe delay dementia. Meditation can improve focus and alertness.

All of those are worthwhile. The top-line, raise basic IQ stuff? To my knowledge doesn't exist.

chrisdone• 9 years ago

I'll cite your Hacker News comment in my next paper.

wannabe_miner• 9 years ago

Can you link to one of your previous papers?

rajadigopula• 9 years ago

I am a lifetime member of lumosity for past few years. they used to have a beta-version game of "Dual-N-Back" which is the only one that might have some impact on brain. The irony is they removed it and keeping the useless ones; even keep modifying the UI of the same games again and again..

gargarplex2• 9 years ago

Dual N Back is a test of memory. Where is the evidence that practicing does anything besides decrease the accuracy of the test?

hoodwink• 9 years ago

Here is the most comprehensive review of the DnB experiments I have ever seen:

https://www.gwern.net/DNB%20meta-analysis

it_learnses• 9 years ago

actually dual n back skills are also non-transferable.

b6• 9 years ago

I have no idea what I'm talking about in this area, but the only brain games I've run into that I think might actually be helpful are the ones that are somewhat painful, e.g., BrainHQ. Definitely not fun. I start to sweat just thinking about some of the tasks in there.

Bartweiss7• 9 years ago

Probably, no brain games at all boost IQ in healthy people. But some of them definitely help with memory, focus, etc, and I agree that those seem to be the hard ones.

Dual n-back, the general benchmark for these things, is hard. It's exhausting like intricate coding or a tough math proof is exhausting. Certainly I'd be suspicious of anything claiming to do better than that with less mental effort.

hmahncke• 9 years ago

I was involved in designing most of the BrainHQ tasks. They're supposed to make you sweat.

kranner• 9 years ago

Title should include "for healthy young adults" as in the study:

"The researchers note limitations in the study, most importantly that they only looked at young, healthy people. Brain-training games may still prove useful for aging adults or people suffering from addiction or unhealthy behaviors, the researchers caution."

SilasX2• 9 years ago

So, kinda like: "Adding a two mile walk every morning was found to have no health benefits[1]

[1] we only studied marathon runners"

abtinf6• 9 years ago

That is an incorrect analogy, because that claim would be about enhancing performance for peak performers.

This study shows that there is no basis for the claim that you can enhance performance of normal/healthy performers.

The researchers state the have insufficient evidence to determine the effect on low/unhealthy performers.

Bartweiss7• 9 years ago

Except - Luminosity has consistently implied and claimed they can boost performance in young, healthy people.

That study of two mile walks for marathoners would suddenly be worthwhile if a popular, well-funded program claimed they could improve your marathon time with a two mile walk.

abtinf6• 9 years ago

"Brain-training games may still prove useful for aging adults or people suffering from addiction or unhealthy behaviors, the researchers caution.

Given the context of the article, this is an unscientific and immoral statement.

It is irrational to believe something is true or possible unless you have evidence for it.

If the researchers have a positive reason to believe brain-training games are a form of medical treatment for aging or addicted populations, they should state that. None of this "may" nonsense.

dragonwriter• 9 years ago

Given the context of the article, this is an unscientific and immoral statement.

It is neither unscientific nor immoral to note a popular belief which the study does not provide a basis for rejecting on which a conclusion might be likely to be improperly formed based on the study.

"May" is not a statement of positive belief.

rodiger2• 9 years ago

It is irrational to believe something is true or possible unless you have evidence for it.

If you have no evidence against it, it is not irrational to believe something is possible.

ncallaway3• 9 years ago

It is irrational to believe something is true or possible unless you have evidence for it.

This is actually two statements.

It is irrational to believe something is true unless you have evidence for it.

Sure, I'd agree with that.

It is irrational to believe something is possible unless you have evidence for it.

Whoa, there. This seems like it goes...way beyond rationality. I would argue that you would need strong evidence if you were going to claim something was impossible.

ebbv• 9 years ago

That's a stretch. There's no evidence to suggest that it will be beneficial for anyone. No matter how many studies you run you can always find a group that isn't covered by the study and say "Well it STILL might be helpful for Red haired, left handed people between 5'2 and 5'3 with 2 siblings of the same gender."

closed3• 9 years ago

Slightly off-topic, but the pushback against brain training owes a huge debt of thanks to Randy Engle (and others), for loudly questioning the methods and rational of the early "become smarter by the magic of dual n-back" research.

Here's an article where he compares it to cold fusion: http://www.nytimes.com/2012/04/22/magazine/can-you-make-your...

michaelcampbell• 9 years ago

Was the claim not "doing n-back helps working memory"? Hardly "become smarter". Whether or not it works is worthy of discussion, but straw-man-ing a hyperbolic claim that wasn't made doesn't add to the discussion.

closed3• 9 years ago

That was half the claim. The full claim was that it improves fluid intelligence by improving working memory. The title of Jaeggi 2008 is "Improving fluid intelligence with training on working memory".

http://www.pnas.org/content/105/19/6829.abstract

codyb• 9 years ago

On the other hand, things like exercise and learning a new language have pretty conclusively shown benefits when it comes to reducing symptoms of dementia and brain function loss. [0] [1]

If you'd like to improve your memory I suspect you'd be far better off studying the masters from Aristotle to Dominic O'Brien whose memory palace techniques seem incontrovertibly proven if you consider the techniques of the winners of the world memory championships. [2] [3]

Finally, meditation is not causally linked to higher levels of grey matter but the research seems promising [4]

(I tried to find decent links but being at work can't spend a significant portion of time on the post, these should be good starting points for anyone interested).

[0] http://www.health.harvard.edu/blog/regular-exercise-changes-... [1] - https://www.livescience.com/46048-learning-new-language-brai... (sorry for the ads, and I think it autoplays a video :( ) [2] - https://en.wikipedia.org/wiki/Art_of_memory [3] - https://en.wikipedia.org/wiki/Dominic_O%27Brien [4] - https://www.sciencedaily.com/releases/2015/02/150205142951.h...

overgard2• 9 years ago

I dont think abstract games like luminosity work, but I think its fatalistic to assume that your intelligence is fixed at a certain point. You can no doubt expand your intelligence (not just knowledge) by expanding your horizons, but learning is a difficult process.

Bartweiss7• 9 years ago

This... feels more like optimism than evidence.

You can improve your performance on almost any single task, which is great news. Certain forms of learning (e.g. language) help defer dementia, exercise and meditation improve focus, memory tricks improve recall, learning diverse topics improves synthesis.

But in terms of 'intelligence' in the clinical sense? Despite many people's efforts, I've never seen a single convincing success for healthy adults increasing their IQ. You can practice a specific test until it's inapplicable, but if you take one test, try to raise your IQ, and take another test, you won't see gains. Dual N-back is still the best thing known, and it doesn't work appreciably.

Fortunately, the practical stuff - memory, focus, breadth of knowledge - is awfully helpful in real life. But I've never seen any evidence at all that intelligence in the IQ sense can be raised in adults.

dwaltrip• 9 years ago

As you hint at in the last paragraph, IQ isn't the most important factor.

What one can actually accomplish is what matters most. If one is able to improve the layer above raw IQ, the fundamental mental models that provide the base for interpreting, analyzing, and synthesizing everything that happens to them, that would provide immense benefits.

It seems to me that is possible, to an extent. This is something I strive for, and I think I have made a bit of encouraging progress. Of course, evaluating this is... uh... not quite straight-forward, to say the least :)

overgard2• 9 years ago

Admittedly I have no evidence, but anecdotally when I started taking my health and diet seriously I feel like I gained about 10 IQ points just from fueling things better. Similarly, learning various mnemonics's and learning strategies has really helped my ability to learn more rapidly. You could argue that I'm simply capitalizing on my innate ability better -- and that would be true -- but the same argument would apply to gaining strength by training muscles. Nobody sees a muscular person thats ripped and thinks they just are lucky to be that way -- we recognize that there is both biology and effort in play, but for some reason people seem to think intelligence is pure biology. I cant prove my case, but im skeptical of that proposition.

No doubt people have different ceilings for their potential, but I feel like most people who are viewed as smart arent just leaning on natural talent, rather they've cultivated skills that get the most out of what they have

jmcgough2• 9 years ago

I spent three years of undergrad working on neurogenesis research - was actually able to show that voluntary exercise could help ameliorate memory deficits caused by a model of a neurodegenerative disease we'd induced in rats.

A Luminosity recruiter contacted me after I left, and I asked for evidence that their research is actually helpful. They were very kind and passed along a number of studies that they'd done. I wasn't super impressed. Among other things, they were marketing their games to the general public, not the populations represented in their small studies. Quite honestly, aerobic exercise causes a huge upregulation of neurogenesis (and many other positive things), so people would be better off going for a light jog 2-3 times a week. It felt like they were taking advantage of families desperate to help a family member with dementia and willing to try anything.

closed3• 9 years ago

Interestingly, one of my old colleagues has research that examines exercise / sport practice in humans as an alternative to the WM training approach. (You two might get along :).

https://scholar.google.com/citations?user=_0GhDagAAAAJ

rayalez• 9 years ago

Can you share some more recommendations or advice on how one can increase neuroplasticity/neurogenesis?

Aside from the diet/exercise/socialization/meditation, what should people be aware of?

Are there some effective supplements/nootropics, or some promising research to look into?

_jtrig• 9 years ago

As far as your question about taking drugs/supplements to increase cognitive function, I can say with certainty any positive effect in the short term will eventually fall to long term dependencies and addictions which will, in turn actually decrease your resting cognitive function.

magic_beans• 9 years ago

Cognitive improvement won't be found in drugs or supplements. The answer is in lifestyle change and recognizing the body and mind as one system.

jmcgough2• 9 years ago

I left academia about five years ago, and don't want to create the impression that I'm some brilliant academic researcher (I spent three years doing rat/staining/counting work in a lab and wrote an honors thesis on the topic). I try to keep up on current research when I can though, the work pretty much consumed my life and I kind of miss it sometimes.

Regular aerobic exercise (stuff that elevates your heart rate) is still the best thing we know of. It's good for upregulation of neurogenesis and BDNF.

There are dozens of papers out there showing that a certain treatment/activity/drug increases neurogenesis. It's funny, I was talking about neuroscience to a lyft driver a way back who was curious about neurogenesis/neuroplasticity, and I told him that exercise is your best bet, but we also had some good results treating our brain-damaged rats with an acetylcholinesterase inhibitor. He shrugged off the exercise bit but of course asked me to spell out ACh inhibitor for him (which I really wouldn't recommend any random person off the street taking).

I experimented for a bit with nootropics after I graduated. Gwern's posts are really interesting if you haven't read him (https://www.gwern.net/Nootropics). A number of people have found success with a basic choline and -racetam stack. As much as tech blogs love blogging about how we're all hooked on nootropics in the valley, it's unlikely that taking some pills in the morning is going to dramatically change your life (though it might nudge it in the right direction). As one of the other replies said, lifestyle changes are still the most effective approach.

eagsalazar22• 9 years ago

This is honestly kind of depressing to me. I've never used an app like Luminosity regularly but I like the idea that I could train my brain and improve my focus and memory. Neuroplasticity right? Well actually no :(

pkghost• 9 years ago

Neuroplasticity yes indeed; Lumosity users are at least getting better at Lumosity games. Generalization no.

eagsalazar22• 9 years ago

It seems surprising to me that people agree you can increase focus and memory through exercise but not IQ. I would think that by increasing focus and memory your comprehension scores, for example, would increase dramatically. I know if I'm distracted my comprehension is way lower. Is that now how IQ tests work?

basseq• 9 years ago

My wife is a PhD in applied cognition (field of psychology), wrote her thesis on improvements in working memory and attention, and is working on an early-stage business around better brain coaching[1]. (More of a 1:1 model than a tech stack.)

She has long stated that Lumosity is crap. For two three main reasons:

1. The "improvement" you see in playing Lumosity games is due to getting good at the game, not actually underlying improvements.

2. The games you do want to pursue need to to be strategic and challenge the mind. They need to be different, and involve complex scenarios. She likes Rise of Nations, by way of example. Flash minigames don't cut it (and neither do crosswords).

3. Brain games are 20% of a total set of activities that have an impact on brain health, the other 80% including physical exercise, diet, socialization, and lack of stress (mindfulness training helps here).

Anyway, fascinating stuff, and glad Lumosity's "bad science" and false claims are coming to the fore.

[1] http://brainevolved.com

seshagiric• 9 years ago

So in general is there any thing that does help improve cognitive performance?

colorint3• 9 years ago

Well, you'll want to start by zeroing in on exactly what you mean by "cognitive performance." The abstract of the study the linked article is about has this to say:

Pre- and post-training, participants completed cognitive assessments and functional magnetic resonance imaging (fMRI) during performance of validated decision-making tasks: delay discounting (choices between smaller rewards now vs. larger rewards in the future) and risk sensitivity (choices between larger riskier rewards vs. smaller certain rewards).

You could certainly imagine doing things to get better at these kinds of tasks---including practicing these tasks. Whether this gets at some kind of murky "cognitive performance" thing depends on how much you're into phrenology and dualism.

Since it seems like you want a scientific answer to this question, you'll always have to confront the realities of experimental design, and because of that, you'll have to nail down exactly what it is you're after. And "cognitive performance" isn't nearly specific enough.

l4xecz• 9 years ago

cmd+f luminosity results in nearly as many occurrences as lumosity. That name is cursed. Probably doesn't help that the correct name gets auto-corrected to the incorrect.

manbearpigg• 9 years ago

If luminosity challenges have any consistency at all, then you would think that users must perform better on luminosity challenged themselves.

Study finds Lumosity users show no improvements in cognitive performance

Related Stories