Increasing Retention Without Increasing Study Time [pdf]

https://files.eric.ed.gov/fulltext/ED505647.pdf

by JustinSkycak • 2 years ago

378 131 2 years ago

files.eric.ed.gov

ngetchell• 2 years ago

[2007]

It's only a four page article about a particular spaced study strategy. Nothing really groundbreaking nor a comprehensive collection of techniques

boredemployee• 2 years ago

any other recommendation for articles regarding techniques that works for most people?

sva_• 2 years ago

The default algorithm in Anki, in case you don't know it

CuriouslyC• 2 years ago

This is pretty much the current state of the art in learning research: https://bjorklab.psych.ucla.edu/research/

kqr3• 2 years ago

The comment I wrote below is weird. What I mean to say is google for articles on writing good prompts. The rest matters less.

-----

Ignore this:

Spaced repetition, if you can stick with it, works. The details around schedules etc. don't matter as much as doing it in a way that you can keep up.

Also writing good prompts is important. I have yet to write more extensively about this but here is a start: https://news.ycombinator.com/item?id=41126734

wahnfrieden6• 2 years ago

FSRS v5 is state of the art

lxm• 2 years ago

https://www.goodreads.com/book/show/19288640-how-we-learn

david_allison2• 2 years ago

Dunlosky J, Rawson KA, Marsh EJ, Nathan MJ, Willingham DT. Improving Students' Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology. Psychol Sci Public Interest. 2013 Jan;14(1):4-58. doi: 10.1177/1529100612453266. PMID: 26173288.

adamgordonbell• 2 years ago

Scott Young has a lot on this, including going deep on the literature and explaining it to populist student audience:

https://www.scotthyoung.com/blog/the-best-articles-on-learni...

For instance, on memory:

https://www.scotthyoung.com/blog/2019/02/15/memory/

highcountess• 2 years ago

It's just typical learning "science" fluff

trwhite2• 2 years ago

I recently read and enjoyed "A Mind for Numbers" by Barbara Oakley, full of lots of techniques for learning maths and science

consf2• 2 years ago

Does the book offer practical strategies?

trwhite2• 2 years ago

Yes, many

bdjsiqoocwk2• 2 years ago

The simplest way of increasing retention is being interested in the subject.

rmbyrro• 2 years ago

The topic isn't about simplicity, but performance and efficiency.

Jtsummers3• 2 years ago

Simple, but not sufficient since you still need to pair your interest with some methods of study and practice. Which methods of studying are more effective or efficient is the question they're researching.

IncreasePosts• 2 years ago

You can just make yourself interested in whatever you want? If not, then it probably isn't the simplest way.

bdjsiqoocwk2• 2 years ago

I would counter, why are you spending time learning something you're not interested in?

wahnfrieden6• 2 years ago

I’ll mention that I’ve gone full-time on an iOS/macOS tool for learning Japanese through reading called Manabi Reader: https://reader.manabi.io

It combines reading and flashcards, such that it tracks every word and kanji you read and learn in order to show you analytics on what you need to learn to be able to read something or achieve JLPT goals, and highlights unknown/learning words in texts. Next up for the flashcard part is to replace the SM2 algorithm with FSRS for the flashcards, as well as having flashcards get passively reviewed simply by reading content.

I also suspect people are missing out on speed of learning when reviewing flashcards one at a time for hours. Besides actively recalling reviewing flashcards passively while reading, I’ll experiment with other review techniques like seeing a page at a time of vocab / revealable answers. Our minds absorb at the periphery of our vision and scanning/inputting a bunch of information at once, too. I’m unconvinced flashcard UI is the final expression of forgetting curve research based learning apps

Also working on Reader features such as manga/pdf/youtube/game emulators, plus expanding to all languages.

shahzaibmushtaq7• 2 years ago

"Teach others what are you trying to teach yourself better" is the best long-term retention strategy in my personal experience.

wahnfrieden6• 2 years ago

For language acquisition?

jakderrida• 2 years ago

Sí

shahzaibmushtaq7• 2 years ago

Also try for other things that are difficult to grasp.

0823498723498723• 2 years ago

For language acquisition: find a partner who speaks the other language and wishes to learn yours, then converse with each of you using your L2, falling back to L1 only when circumlocution fails or to make corrections.

(then how do you practise listening to your L2? Eavesdropping, ancillary encounters, news, movies, etc. give plenty of opportunity, but speaking requires you to make an extra effort.)

kqr3• 2 years ago

Also possibly the most time consuming. Best and efficient with time are two different metrics.

practicemaths• 2 years ago

What is consider efficient if it yields less best results?

I understand time constraints and end goals may prohibit the 'best' approach.

What I do not understand is how can you say something is more 'efficient' if the yield in understanding is less than what you would get with another method.

Hopefully this will clarify my thought process here:

If 'Best' is to teach others and requires 10,000 hours to yield 90-99% understanding.

In contrast, 'Efficient' method requires 2,500 hours to yield 30-40% understanding. However, there is diminishing returns meaning that doubling your hours to 5,000 does not return you with 60-80% understanding, rather maybe closer to 50-60% understanding. With 7,500 hours closer to 65-70% and 10,000 hours may around 75-89% understanding.

Here you've spent the same amount of time but did not achieve the same level of understanding. I think you may have a dynamic 'Best' vs 'Efficient' curve and to switch between those options to optimize maximizing your level of understanding in the least amount of time.

shahzaibmushtaq7• 2 years ago

Yes, possibly the most time consuming if you are thinking of writing a book.

How effective something is comes with what is best and compromise effectiveness for efficiency.

consf2• 2 years ago

Efficiency and effectiveness are indeed different metrics. And I think balancing time and retention needs is key

hinkley5• 2 years ago

Someday you will die. Then you will die again when people forget you. Only the effect you’ve had on the world, including teaching others, who pass that along, outlives that second death.

So what’s this about efficiency? You should be more worried about effectiveness, not efficiency. Particularly in this profession.

cainxinth• 2 years ago

“See one, do one, teach one”

shahzaibmushtaq7• 2 years ago

Yes, monotasking is the way to go.

cyberax• 2 years ago

"Teach others what are you trying to teach yourself better"

A joke in Russian universities:

- A teaching assistant tells a student: "Look, I've been explaining it to you for so long, that I myself understood it!"

hinkley5• 2 years ago

I tried to help some dude understand integration by parts, which I’m not sure I’d understood myself. I warned him. But it was late, he was desperate, and I was just going to go play computer games anyway.

I figured it out, but I’ve no idea if I got him sorted out for his final.

0823498723498723• 2 years ago

A joke in American universities:

The professor is interrupted in the middle of his lecture by a student who asks about one of the previous week's problems:

— Could you please work an example for us?

— No problem!

The professor writes a problem statement, then scratches his head, tugs on his collar, rubs his chin, and at last scribbles a complicated integral on the board.

— There you go!

— Excuse me, professor, but could you please be a little more explicit?

— What do you want from me? I just did the problem 3 different ways.

huhtenberg• 2 years ago

The expression goes "to know is to be able to explain".

riiii2• 2 years ago

Yes, but there are plenty of exceptions. I know brilliantly knowledgeable and extraordinary clever people that are so socially awkward that they can't explain virtually anything.

rqmedes• 2 years ago

To be people who also understand it

hinkley5• 2 years ago

I push the new guys to fix project documentation once they’ve figured out a tricky bit. It helps solidify their knowledge, and helps us double check that they understood, and it’s something they can contribute when they still haven’t become part of the bus number on anything yet.

That all sounds reasonable and smart, but the real reason I do it is the Curse of Knowledge. People in a system can’t see it from the outside. They make assumptions, use opaque or even misleading jargon, and employ circular logic. The new guy doesn’t know the lingo, or the circular logic. Their explanation will make more sense to the next hire than anything I can say. And having it written down this way can also give me new perspective on the system. Maybe it doesn’t have to work this way.

zoogeny2• 2 years ago

Adding a new smart person to the team is one of those golden moments for a team, IMO. You get a tiny window of watching them struggle until the tribal team knowledge seeps into them by osmosis. During that time they don't yet know who to ask the questions to directly so they will post to team slack channels or the lead directly.

One must capitalize on this brief period because smart programmers are flexible and adaptable. Very quickly they will acclimatize themselves to the mess that surrounds them and they will become as blind to the deficiencies as the rest of the team.

specialist• 2 years ago

Nice. Somewhere I picked up the phrase "problem of assumed knowledge" for the phenomenon you describe.

But your phrase is spot on, well defined, and the subject of ongoing research.

https://en.wikipedia.org/wiki/Curse_of_knowledge

Cheers.

godelshalt• 2 years ago

Sounds great but I was expecting more evidence. They talk about this shuffling method, which interleaves material as a way to provide spacing and reintroduce material. But it appears to just be their opinion that it will help improve retention. Also what is the deal with this "hypothetical interaction between ISI and RI"? Why not do enough experiments to actually plot it out? Anyone can graph out a hypothetical interaction.

kqr3• 2 years ago

I think Accelerated Expertise comes to the same conclusion and references the studies to back it up. It's a good book!

abecedarius3• 2 years ago

Years ago there was a great extended comment here on HN about teaching a linear algebra course using, among other tactics, a similar spacing of the homework problems. He reported excellent results. I wish I remembered the username. Ben something?

ExciteByte• 2 years ago

I think it was Ben Tilly and I think his user name is btilly. I also remember that comment you're referring too.

BJones12• 2 years ago

https://news.ycombinator.com/item?id=818367 ?

mrandish• 2 years ago

I really appreciate HN. There aren't many public online forums were one could ask about a user comment made on a related post posted years ago and in less than 20 minutes get two different responses linking to the comment from 15 years earlier.

Perhaps it helps the topic was "increasing retention". :-)

sn9• 2 years ago

He also later expanded it into a blog post: https://bentilly.blogspot.com/2009/09/teaching-linear-algebr...

michaelcampbell6• 2 years ago

The "Make it Stick" book https://www.amazon.com/Make-Stick-Science-Successful-Learnin... cites many many studies in this arena if you're interested.

jppope• 2 years ago

Real question. Does this count as a published paper?

david_allison2• 2 years ago

Yes, with 263 citations

Rohrer, D., & Pashler, H. (2007). Increasing Retention Without Increasing Study Time. Current Directions in Psychological Science, 16(4), 183-186. https://doi.org/10.1111/j.1467-8721.2007.00500.x

layer8• 2 years ago

It was published in a scientific journal, so yes.

moffkalast• 2 years ago

Because people forget much of what they learn, students could benefit from learning strategies that provide long-lasting knowledge. Yet surprisingly little is known about how long-term retention is most efficiently achieved.

I've always thought that the real problem is information relevancy. People need practical uses for remembering something beyond synthetic bullshit exams. Efficient techniques are great, but nothing demotivates more than not having a reason to learn beyond being told that you have to.

Nobody has to reach for flashcards, extensive notes, or advanced techniques when trying to learn something they're actually interested in, it's retained almost immediately and effortlessly. There has to be some kind of subconscious gauge of information relevancy that physically controls the level of absorption, a sort of "learning rate" if you will.

larsrc• 2 years ago

I disagree. Motivation will keep you reading the material, but that alone will not make you remember it that much better.

When you're motivated about something you can practice, you may remember more by doing it (which also acts a spaced repetition). But good luck trying that with, say, astrophysics or macroeconomics.

When motivated, you're also more likely to pick up other books on the subject, which is another form of spaced repetition.

james-revisoai2• 2 years ago

I agree motivation to learn and maintaining interest is key, to effect the learning rate you mention. The enjoyment itself, although a crucifying word to use in discussions when teaching students considered the "best of the best", still matters and predominates over the effectiveness factor for most of the semester for students, at least. Because of just being "told to learn" it.

Basically 70% of the semester most students are not studying 40 hours - they are doing 30 hours of real work, and perhaps only 15 hours effectively. And for good reason: Nothing is bridging or rewarding them in a way that interests/motivates them, for courses where the interest isn't natural.

A bridge to motivate them would be ideal. In 2021 I started using GPT-3 to generate motivational "reasons to learn a concept" cards for my flashcard app, - Revision.ai - which you can read about here in the 3rd item: https://www.revision.ai/articles/20ThingsRevisionAIDoesForBe... - the reason we disabled them was simple: we could never quite time the cards right to help the student when they needed it. When the app is closed, they aren't motivated - they don't see them. Mid study session? Showing such cards (or AI generated examples) interrupted the flow [https://www.instagram.com/p/CVVlIuVg31W/] We have also tried recommending relevant short/mid length youtube videos for visual/"breaks" from overwhelming learning. That did not boost student success either. I guess it doesn't address that you are still being told to do it, not naturally flowing into learning.

I welcome any technical or conceptual ideas you have to improve this and help increase interest amongst students. We have found that turning lecture slideshows into sets of exercises with clear visuals[https://www.instagram.com/p/C5ByftwiJ00/], breaking content up, and showing progress does motivate students to study more(and in a semi-related dissertation I wrote, possibly reduce Test Anxiety and tension). Please let me know of any ideas you have!

WaitWaitWha• 2 years ago

From the abstract (emphasis added):

Because people forget much of what they learn, students could benefit from learning strategies that provide long-lasting knowledge.

What learning strategies were taught in your secondary educational years?

tnvmadhav• 2 years ago

spaced repetition

aflukasz2• 2 years ago

If you are interested in similar research, take a look at https://supermemo.guru (somewhat hidden "about" page: https://supermemo.guru/wiki/SuperMemo_Guru:About). Author spent quite some time on the topic and its history, including writing applications.

light_hue_1• 2 years ago

A lot of what's on that page is just nonsense. The page has a very clear bias toward one method and writes all sorts of unscientific drivel as a result.

https://supermemo.guru/wiki/School_damages_your_brain

As someone who also publishes in neuroscience I can tell you this is total and utter trash.

aflukasz2• 2 years ago

I'm not in a position to judge this author's work as a whole, as this is not my area of expertise.

I know this person as an author of "super memo" algorithms for spaced repetition learning, so yes, he most likely have a bias, probably even financial stake to some degree, although not sure, I don't follow him in any way, so don't know if this ever was or still is the case.

In the past, I've used some of his algorithms in particular, with quite interesting results - subjectively speaking, not in any scientific regime, And, obviously, those are just one of many. I also remembered this was his area of interest in general, so assumed his wiki to be of some interest in the context of discussed article.

Thanks for your critical opinion, it will contribute to my priors about his writing.

MichaelNolan• 2 years ago

I enjoy browsing r/Anki from time to time, and that site is frequently mentioned there. Spaced repetition is definitely a real thing, but the rest of the SuperMemo site is a bit “crazy”. The author does have a “20 rules” for good card writing which is pretty good advice though. http://super-memory.com/articles/20rules.htm

aledalgrande• 2 years ago

Why do researchers need to use acronyms for a 2 pager article?

cod1r• 2 years ago

Sometimes I feel like I know nothing because information goes in one ear, stays for a bit, then just runs out the other ear.

Then I have to go back and review.

Repeat.

Modified3019• 2 years ago

One thing I’ve found when I spend most of a day trying to learn things, is that having a ~1.5 hour nap is pretty important. Not sure what goes on, but it feels like it turns the disconnected “fog” of information in my head into something that is a least loosely linked and can be vaguely recalled upon rather than completely forgotten.

Unfortunately I kinda need to plan these sorts of days ahead, because trying to nap past 2PM is a good way to fuck my nighttime sleep.

michaelcampbell6• 2 years ago

I'm pretty sure the current thought is that you ONLY "learn" during sleep, if you consider learning to be filtering, organizing, and throwing out vs keeping.

mrlase• 2 years ago

I'd highly encourage anyone interested in the contents of this article/learning about learning to read through the submitter's blog: https://www.justinmath.com/blog/

james-revisoai2• 2 years ago

I also recommend https://www.learningscientists.org/posters from scientists in the field, which covers additional scientifically-effective approaches that a course designer or more long term approach might take. For example, not only applying Spaced Repetition, Interleaving and Active Recall (all possible through automated spaced repetition apps based on simply input), but elements like Dual coding (related to "Varied practise") - mixing visual and other elements when learning (which requires effort to create them), and elaboration practise (like free recall). To the end of applying these - and other critical elements like focusing on the motivation of the learner and enabling them to understand what they are missing with progress & identified "blindspot" misconceptions (on their own incredibly powerful), I've been developing Revision.ai since before GPT-3, through a Psychology MSc.

hnthrow12390• 2 years ago

Content is good but he is part of a paid math program and some articles and submissions appear to be indirect advertising/marketing if you think about it. (but it's effective and less forceful) https://www.justinmath.com/why-is-the-edtech-industry-so-dam...

lupire3• 2 years ago

Yes. The submitter uses spaced repetition techniques to build persistent memory of his spaces reposition math education business.

chrisweekly2• 2 years ago

See also the wonderful https://betterexplained.com

paulpauper• 2 years ago

High IQ. A component of IQ is memory--working and long-term memory. There is a reason why doctors are smarter than average--you got to be smart to memorize all that stuff. I am skeptical that hacks make much of a difference. There is also a difference between recall and understanding. This is why speed reading courses are dubious because you're not really understanding things, but just recalling items in the text.

westcort• 2 years ago

The time between study and testing (retention interval) is the most salient factor for students. Repeating material at intervals attenuates the forgetting curve somewhat, but not enough to generate passing scores. Cumulative exams only force students to study all material in a massed study session prior to a final exam.

I say this as someone who works on education and as a lifelong learner who has tried many ways of improving retention.

Nevermark3• 2 years ago

This is not a critique of the (nice) study, but what seems to me the overall context that should be kept in mind, especially once we are talking about "optimizing".

Optimizing "study, retention phase, test" for greatest knowledge retention at a delayed test time, is very different from optimizing for greatest value of knowledge learned.

To optimize learning value, learn things that are immediately useful, you can immediately incorporate into learning something else, and ideally both.

The sooner and more you use something, the greater its value AND the greater your retention will be.

If you have to learn something valuable but with no short term use (how to handle a rare brain surgery complication), find a way to use it. Create an ongoing useful project that will revisit that knowledge during the "retention interval" (e.g. a concise summary of rare situations you need to handle, for you and others, that you can revisit and improve with additional and updated knowledge).

So optimize "topic choice", "topic progression", and "study, (optionally) test, use, use, use", for total value of learning.

"Use" is motivation, test, study review, and value realization put together.

zeroCalories2• 2 years ago

I think a lot of topics are much less sequential the further you go. As an adult I spend most of my time repeating the fundamentals of my field, and learning a topic deeply as needed. For children it probably makes sense to cram the multiplication tables.

Nevermark3• 2 years ago

I would think that the opportunities to immediately "use" multiplication, instead of just practice it for tests, or some future numerate citizenship, would be omnipresent.

If you don't use something after you learned it, you miss out on:

1. Learning how it is actually applied

2. Discovering how the knowledge is useful for you personally, in ways you may not expect if you don't actually experience using it

3. Deeper understanding and mastery of the knowledge

4. Much much much better retention

It is worth creating some immediate use for new knowledge, even the smallest possible useful or creative project, for better retention alone.

learning a topic deeply as needed

That is the ultimate use-driven learning model.

As for non-sequential, I agree. The more we manage our own learning, the more it is a directed graph (i.e. prerequisites translate to many follow up paths), and eventually just graphs (many ways to order topics, and alternate combinations of prerequisites for each topic, in any complex area).

formerly_proven• 2 years ago

Basically all undergrad stuff is fundamentals and "optimized cramming" (we call it bulimic learning, because you stuff knowledge into you just in time to throw it back up in the test) means tons of people "achieve" degrees without being even dimly aware of the existence of the fundamentals afterwards. If you have a bachelor's degree in e.g. computer science, the idea of encoding data shouldn't be an unknown unknown to you.

shortrounddev23• 2 years ago

They don't teach multiplication tables anymore as far as I know. I didn't learn them and I was born in 1995

michaelt• 2 years ago

I sure hope my brain surgeon isn’t trying to create an opportunity to practice dealing with rare complications!

Nevermark3• 2 years ago

The brain you lose in the name of practice, may save millions!

wahnfrieden6• 2 years ago

That’s why I made Manabi Reader for language learning (currently only Japanese): https://news.ycombinator.com/item?id=41275227

It encourages finding content from native sources, learning new words in context from materials that interest you rather than textbook content, and helps you build your own personal corpus of sentences as you encounter them in the wild. Will soon expand to more media types than just web and epub, to include YouTube/comics/HDMI input/game emulators etc

If you find flashcards from others, you also get easy tools for discovering more sentences from source materials that might interest you or from your own corpus. More coming for this and other word/kanji-level tracking analytics. Offline-first and privacy friendly.

andai• 2 years ago

This is great. Reminds me of the writings of Khatzumoto (All Japanese All The Time). On that note, do you know how to contact him? His site got deleted recently and I have an idea for restoring a static version.

cbracketdash• 2 years ago

[Mods: it might be helpful to tag this paper as written in 2007]

It seems to me this paper is bringing to light the idea of spaced repetition for learning and this has become quite popular in the productivity/learning culture of today (e.g. Ali Abdaal).

Alternatively, mathematics textbooks could easily adopt a format that engenders spacing.

I tutor middle school students in mathematics and this is definitely being implemented in their textbooks! At the end of each chapter, there a is normal chapter review practice test followed by a "Cumulative Practice" which reviews topics from the previous chapters in the book. These are especially beneficial to my students as, like the paper highlights, it promotes long-term memory of those topics.

For example, although computer-based instruction typically provides extensive retrieval practice and rapid feedback, it offers a currently unexploited opportunity to schedule study sessions in ways that optimize long-term retention.

There is an immensely popular software called Anki which implements exactly this "spaced repetition"-type protocol.

__float• 2 years ago

Anki is very much... an expert's tool? I think there's quite a lot of opportunity (not necessarily monetarily :)) for SRS software with a good UX.

Anki's data model is rather strange, which partly relates to its flexibility but has some unexpected downsides and trap-doors. I would love a slightly more opinionated SRS tool! (Maybe with a bit more UI polish, rather than rewriting the backend in Rust)

davisoneee• 2 years ago

I don't see how anki is an experts tool...unless you want it to be.

You can simply dump everything into a single collection, use basic 'front' and 'back' cards...and review. Yes, it may not be 'optimal' depending on how you mark cards, but for most people, it's good enough. Better than nothing, and probably an improvement as an addition to whatever they are currently doing.

If you want expert features, you can. You can drill, tag, organise, create different templates....but none of that is really necessary, or particulary pushed on you as something you should or have to use.

easton• 2 years ago

Quizlet has a spaced repetition mode you can use, you might need to pay for it? I used it a few years ago for some stuff I needed to memorize in school.

michaelcampbell6• 2 years ago

Anki is only an expert's tool if you fall into the all too common trap of trying to optimize everything by microtweaking the settings.

I mean, sure, if that's your thing and you enjoy it, go for it. But it is not necessary, and (SM2 vs FSRS arguments aside), it works right out of the box with the default settings and algorithm.

Like anything, trying to get that last 10% optimization is WAY more work than getting the first 90%. But "using Anki" (in any way) is such a multiplier over NOT using it that just taking that step is enormous.

michaelcampbell6• 2 years ago

SRS software with a good UX.

This is a common complaint, but feels like a "delicate flower" argument. The UX is pretty spartan, but a fancy whiz bang UI isn't necessary to remember facts.

Not everyone needs all that, and the people that do (which is fine, btw), is a reflection of the person more than the tool.

nicomeemes• 2 years ago

There is a comparable software that has a friendlier UX: https://mochi.cards/. It's basically Anki, if Anki were smoother. Does cost a tiny bit though.

sunnybeetroot• 2 years ago

Mochi is excellent and the dev is responsive.

krehwell• 2 years ago

well. Knowt (knowt.com) offers spaced approach too. I love it better

ordu• 2 years ago

I'd guess, that overlearning is so widespread not because it is beneficial for an individual, but as a way for a teacher to deal with a lot of students at once. To measure just the right amount of learning the teacher needs to work with each student individually, and give exercises for them based on a quality of their newly formed knowledge. But the system is mostly tuned for mass-education to amplify the impact of a teacher's work.

Increasing Retention Without Increasing Study Time [pdf]

Related Stories