Andrej Karpathy on learning (should never be fun)

https://twitter.com/karpathy/status/1756380066580455557

by HiPHInch • 2 years ago

75 57 2 years ago

cen4• 2 years ago

Actual Teachers don't just know a subject, they know something about Pedagogy.

And Pedagogy is all about getting the student into constant practice rituals until they have mastered the subject. Without practice there is no education/learning.

Herbert Simon - "Criticism of practice called "drill and kill" is very common. Nothing flies more in the face of the last 20 years of research than the assertion that practice is bad. All evidence, from the laboratory and from extensive case studies of professionals, indicates that real competence only comes with extensive practice. In denying the critical role of practice one is denying children the very thing they need to achieve real competence. The instructional task is not to "kill" motivation by demanding drill, but to find tasks that provide practice while at the same time sustaining interest"

vacuity• 2 years ago

There is something to be said about exactly what the practice is/how the practice is structured. Both the learning and practice phases should rely less on rote memorization and more on dissecting and understanding principles. For instance, physics without basic calculus means being handed a set of opaque equations, which encourages blind application and needless confusion where that doesn't work. At least in the US, I find this lacking.

namaria3• 2 years ago

Realizing entertainment driven content is not an education is not insightful and does not warrant the dichotomy that actual education should be 'not fun'.

Rants on social media are an easy outlet for frustration and should not be conflated with insightful opinion piece journalism.

christophilus2• 2 years ago

I think it’s a useful bit of commentary. It’s a nudge. I think there’s plenty of entertainment-driven content that masquerades as educational. Andrej is giving a rule of thumb to help people identify when they’re being lulled into an entertainment trap while thinking they’re doing something worthwhile (not that an occasional escape isn’t worthwhile). I think his veggie straw analogy is a good one, and I’m filing it away for reuse.

namaria3• 2 years ago

It's not bad advice. But it is also not insightful and might be harmful. No one should be worried that they're having too much fun when they are learning.

forkerenok• 2 years ago

He caveats this in the post, so he's not really dichotomizing.

IIUC, he implores educators to optimize for productive "workout" rather than fun. From my point of view, "workouts" must be rewarding on average, but need not be fun.

MattHeard• 2 years ago

It doesn't have to be actively not fun either, but the primary feeling should be that of effort.

Karpathy clearly wrote something different that what you say he wrote.

namaria3• 2 years ago

Selective quoting is fun!

Here's my take: "Learning is not supposed to be fun."

"Not supposed to be fun" has a clear meaning. "It doesn't have to be actively not fun" doesn't have a meaning. No one ever says "actively not fun". It's a cop out to pretend like he's not presenting a false dichotomy, and I can see right through it.

danlitt• 2 years ago

"should be effortful" and "should not be fun" are not opposites -- they aren't even related. Most people who "reach the top" of particular domains do so because they get enjoyment out of the required practice.

There are also lots of valuable intellectual tasks that appear more fun than effortful at face value. Watching talks at conferences springs to mind.

vanjajaja1• 2 years ago

i made a similar reply, linking to the ideas of flow state etc [1]. i dont know why you'd chase the feeling of effort, surely you'd avoid the feeling of effort where possible without reducing learning

[1] https://x.com/vanjajaja1/status/1756426000613994952

seizethecheese• 2 years ago

Most people who "reach the top" of particular domains do so because they get enjoyment out of the required practice.

Citation needed. There are many anecdotes of peak performers not liking the practice. I guess we could all agree that being at least able to tolerate the practice must be a necessary condition.

exe34• 2 years ago

I've often spent days figuring something out of spite rather than enjoyment. I think people have all sorts of motivations.

nemoniac• 2 years ago

“I hated every minute of training, but I said, 'Don't quit. Suffer now and live the rest of your life as a champion'.” ― Muhammad Ali

christophilus2• 2 years ago

Learning is not supposed to be fun. It doesn't have to be actively not fun either, but the primary feeling should be that of effort. It should look a lot less like that "10 minute full body" workout from your local digital media creator and a lot more like a serious session at the gym.

I don’t think the author disagrees with you. Some people really enjoy the effort of a tough workout. But they don’t do it primarily for the fun. Can / should workouts be entertaining? Sure. But there’s a difference between a primarily entertaining bit of content and a primarily workout-oriented bit of content.

salomonk_mur• 2 years ago

Completely agree. As commented elsewhere, the main driver in true learning is practice.

I also wholly reject the idea that this "edutainment" content is useless. Same as spaced repetition, once you see some concepts often enough (which the algorithm guarantees if it knows you are into that topic), you will learn something. I don't consume such content myself, but to dismiss it just looks as rejecting formats different than what you used.

sunrunner• 2 years ago

spaced repetition, once you see some concepts often enough

I think there's a quote on this but I can't remember by whom, but I think it's something like "Familiarity doesn't lead to repetion, repetition leads to familiarity."

Even if not a quote, I think the idea (with spaced repetion adding the concept of time) of repeated exposure to a subject as the thing that drives understanding, and not understanding just being a thing that itself allows people to do the subject a lot, is an important one to keep in mind for learning any new topic, particularly ones that are perhaps further away from your current knowledge base, and so feel inherently trickier.

And if you fully comfortable with the subject, you're probably not learning anything at that point ;)

OutOfHere2• 2 years ago

Practice is the boring person's substitute for a diversity of real projects that interesting people work on to strengthen their learning. If you are boring, you can't be helped.

ignoramous2• 2 years ago

"should be effortful" and "should not be fun" are not opposites

Karpathy doesn't really say they are. What he's postulating is that learning must be purposefully effortful. Kind of like playing any sport, say Football / Volleyball. You'll enjoy it, but it isn't without effort.

tim333• 2 years ago

Thinking of learning skiing rather than academic stuff I think fun is quite a good guide. If it's too easy you don't learn much and kind of get bored, too hard and frustrated or injured but the most fun is a bit challenging where you learn new things but not too hard. I think out instincts may be more evolved for physical skills than academic ones.

zigzag3122• 2 years ago

It's possible to spend a lot of effort, without learning anything. So, effort is not the goal.

dr_dshiv• 2 years ago

The fun needs to come from the intrinsic motivation of the learner, not the content.

Good quality learning materials can be fun if you want to learn — or miserable if you don’t.

I learn way more when I’m personally driven than if it feels like “I have to.” But that’s me.

cloogshicer3• 2 years ago

The most surprising thing I learnt from the past 5 years of full-time teaching both complete beginners and experienced programmers is this:

The main barrier to learning, for most people, is emotional, not intellectual.

Because of that, I disagree with most of what Karpathy wrote here.

homarp• 2 years ago

can you elaborate on the 'emotional' part?

samarthr1• 2 years ago

not GP, but, there are times when I just dont have the "will" to do anything "useful", when I just want to veg out, and be passive (as in , I am willing to listen to something, but not in the frame of mind to process anything)

(Or)

Another interpretation of GP is the mental barrier that "X Topic is very hard", and I am not _that_ smart, so I dont want to fail?

cloogshicer3• 2 years ago

Sure!

A lot of students have feelings of self-doubt. They get easily frustrated, or bored, or impatient. They feel like they're not good enough, or that they will never get this. They get stuck in negative thought spirals like that. They don't dare asking questions because they've been taught that it's a sign of weakness. etc.

These feelings are often entirely disconnected from reality, even the best students have them.

Because of that, I think any piece of educational content needs to tackle the emotional side as well, maybe even primarily. So I do see the point of fun in learning, if it can help cross these emotional barriers.

H8crilA4• 2 years ago

But you're agreeing? Emotionally it is easier to listen to 10 podcast episodes talking about LLMs, rather than to code a simple one yourself. Yet both can take a similar amount of time.

cloogshicer3• 2 years ago

I guess my point is that for some (maybe most) people, doing the latter is just not an option without a lot of guidance and, possibly, introducing some lighthearted elements and fun as well.

H8crilA4• 2 years ago

One key feature of learning is that it must involve some form of failure - at least in the sciences that HN is typically involved in. It is impossible to fail at watching a video, but it is possible to fail at deriving a formula yourself on a piece of paper, or at writing a program that is supposed to achieve some goal. The failure and subsequent recovery is exactly where knowledge is gained. This is perhaps why writing code is so much fun, the failure-recovery loop is very short.

viraptor3• 2 years ago

(citation needed)? The more general knowledge of the related area you have, the more the new information starts looking like "this other thing, but with a slight twist". But that means you don't need to fail to learn. If you tell me about some new networking/routing layer for example, I don't need to "fail" in any way to understand how it works - it's going to be like or unlike something we already have.

H8crilA4• 2 years ago

At some point you'll have to use your knowledge for something, and only then you'll find out whether or not your knowledge is complete, i.e. whether or not you've actually learned the topic, or is there an entire aspect you haven't thought about before.

Of course this is all a simplification, if you learn protocol X and then someone comes up with protocol Y that just adds a few fields then it may be just enough to read about these new fields. I'm talking about larger stuff, like knowing quite a bit about radio communication protocols and encountering secure wireless ranging protocols. A whole other game.

Wieldable4640• 2 years ago

I agree with you, although I would put it differently.

Learning requires feedback, not necessarily failure. The issue with watching a video is not that you can’t fail, you can certainly fail to get the point made in a video, but as you can’t get any feedback it would be hard to figure out that you didn’t get it.

In my view the difference between watching a video, vs trying to implement an idea as code is the direct feedback you get. If you’re wrong or made a mistake, you’ll know and can adjust your mental model accordingly.

It’s also what I like about writing code, if a colleague and I disagree we can just design a test together and find out and learn together.

jacknews2• 2 years ago

I don't think there's anything wrong with edutainment videos themselves, but you only really learn by doing, so they should facilitate that.

For example, my kids watch 'how to draw' videos on youtube, but they pause them every few seconds to copy the work, and then watch the explanations of how shadows work and so on.

There is some research on the effectiveness of videos in education, and how to use them properly (just watching is usually not enough), I'll try to dig up refs.

Draiken3• 2 years ago

That's exactly what he suggests doing. Taking the time, pausing, trying it out, etc. What you're describing is how he believes videos should be produced/watched, and not a 10 minute "5 tips for drawing like a master" video.

I see myself watching a lot of edutainment videos that, if asked about them a week from now, I won't know any of it. Regardless, when I was watching it, it felt like I was learning something, which is very far from the reality.

jacknews2• 2 years ago

Where is he saying that?

He says "seek the meal - the textbooks, docs, papers, manuals, longform."

I'm saying you can actually use some of those edutainment 'learn X in 10 minutes' type things, if you actually practice the thing. And especially if you hit problems or have questions, which will lead to a deeper exploration.

peter_d_sherman• 2 years ago

"Learning is not supposed to be fun. It doesn't have to be actively not fun either, but the primary feeling should be that of effort."

This idea is not wrong; however, I would rank 'persistence' (the willingness and ability to engage in learning down a specific path during many days, many weeks, many years, etc., etc.) as being higher in a "hierarchy of necessity for learning" than that of effort.

And I would rank 'unlearning' (the willingness and ability to "let go", become looser, less intransigent, entertain other opionions, other points of view, of what was previously learned down a specific path during many days, many weeks, many years, etc., etc.) as being higher still in a "hierarchy of necessity for learning" than even that of 'persistence'...

Finally, at the pinnacle of such a "hierarchy of necessity for learning" (should one ever exist!), I would rank 'Imagination' itself (AKA "the goal sought after", or more simply "The Goal!") as the highest tier, the pinnacle of such a "hierarchy of necessity for learning"...

(Also, if effort is to be made, it should be desired effort as opposed to undesired effort... i.e., effort coerced by other people and/or systems...)

But -- that's just me... :-)

Those above values are just mine and mine alone... one insignificant man's humble opinion...

Other people can and will have other "what's necessary for learning" hierarchies; other sets of values, as pertain to education as applied to themselves and groups of other people...

https://en.wikipedia.org/wiki/Maslow%27s_hierarchy_of_needs

viraptor3• 2 years ago

I can't read that and not think about all those influencers and efficiency coaching videos. "Do this thing that works for me, a person addicted to structured functioning." This has so many preconditions about what you want to achieve and what the context is.

For example I've done beginner's programming lessons with kids. Using Scratch, at the level of explaining what a variable is. There were 2 kids who would consume any knowledge given out of pure interest and everyone else who either was entertained while learning or immediately switched to playing ready games from the Scratch library. Anything close to "you're going to do learning/mind workout for the next hour" would just alienate all of them.

That's without even getting into how funny this idea would be to people with executive issues (which we have a decent representation of in tech) "allocate 4 hours" haha.

Don't get me wrong, I'm glad it works for him and if anyone finds it useful/motivating, I'm happy for you. But presenting this as something universal really annoys me. To be a bit more positive - any workout you actually do is better than the one you don't. If you need fun to continue, that's awesome. If you like a challenge, awesome. If you like goals, awesome.

ignoramous2• 2 years ago

https://archive.today/KUQzP

croes• 2 years ago

Where does it say learning should never be fun?

Learning is not supposed to be fun. It doesn't have to be actively not fun either, but the primary feeling should be that of effort.

hamilyon2• 2 years ago

I am trying to learn French to some decent level. I have every anki app, one-on-one coaching, textbooks, podcasts, online courses, and friends to help me. Even LLMs now.

God, it is painful. I think of myself as motivated and talented. It is a torture. It is rote and repetitive to no end. I feel helpless all the time. Progress is slow. Just nothing helps besides considerable concentrated effort. Not fun. Fun XOR progress.

nunez2• 2 years ago

Why is it painful? Why are you learning French? Why now?

phrenq• 2 years ago

I know Karpathy explicitly says that learning doesn’t have to be not fun either, but it reminds me of the book “A Theory of Fun for Game Design” by Raph Koster [0], which offers a counterpoint.

It’s been a while since I read it, but Koster essentially proposes that fun literally is learning at an evolutionary brain chemistry level. That the games we play as children prepare us, in a no consequence environment, for the problems and dangers that we’re going to face as adults. The thing that makes games “fun” is the solving of puzzles and mastery of problems, and we’re much better at learning when it’s fun.

[0] https://game-studies.fandom.com/wiki/A_Theory_of_Fun_for_Gam...

davidivadavid• 2 years ago

I think the game design angle is the best here. What he's gesturing at, to me, is the concept of difficulty as encountered in games. You have to play at the highest level of difficulty you can tolerate without getting frustrated. If it's too easy, it's boring, and a sign you're not learning anything. If it's too difficult you give up. So it's not so much a binary thing as a matter of "right degree of difficulty"/balance.

agumonkey• 2 years ago

There's something important about the limit between fun and not fun in learning. With time you know that many times, stuff that appeared hard and not fun were indeed so simple yet so foreign.. the pain came from trying wrong path for too long.

moogly• 2 years ago

So for those who actually want to learn. Unless you are trying to learn something narrow and specific, close those tabs with quick blog posts.

OK. <closes Twitter tab>

returnInfinity• 2 years ago

A whole lot of works to say Infotainment

indigo0086• 2 years ago

Even more than the shoetification is the infantalisation of teaching online, especially in the software field. Lessons and presentations peppered with cutesy art or memes or images completely unrelated to the presentation adide from silly jokes or attempts to add a warm childlike veneer to what the adult presenter is presenting. Its causes my attention to be split and wondering who the lesson was even made for

valval• 2 years ago

I don’t disagree, but gamification of practice sessions isn’t a bad idea. I’ve always been very bad at focusing on some tasks and sucked at school.

Nowadays I can force myself to learn topics that I don’t care too much about by dividing it into tasks, and “levels” that leads me to feel that I’m making progress on a classic progress bar type interface.

LLMs are pretty good for this, they help a lot with coming up with targets and such.

nunez2• 2 years ago

A) Incredibly rich coming from a guy whose net worth is tied up in AI tech that is used heavily to summarize things under the guise of "learning"

B) Also incredibly wrong. It is absolutely possible to educate in an entertaining way; loads of teachers do this daily! See also: Duolingo; early education learners; many educator YouTube channels

C) The gym analogy is also incorrect. The likelihood of someone dropping out from an exercise program skyrockets if they see it as a slog. It is possible to find ways of exercising that are fun; in fact this is the ONLY way to exercise in a long-term way.

zigzag3122• 2 years ago

Learning is not supposed to be fun. It doesn't have to be actively not fun either, but the primary feeling should be that of effort.

The important thing is the result. That the person learning has learned the topic.

Which is better, learning with great effort or with low effort, if the quality of learning remains the same?

HarHarVeryFunny• 2 years ago

I think the real issue is that you can't learn to DO just by watching, regardless of whether the watching is entertaining/fun or not.

If you just consume educational content, without trying to use it (and learning from the failures and successes than ensue), then you are at best just memorizing, or at worst just entertaining yourself.

In order to learn how to actually DO something, whether mental or physical, you need to be actually DOING it yourself. The more you practice, and overcome failures, the better you will become.

The main reasons for needing to learn by DOING are that:

1) No author/teacher is going to 100% describe all you need to know. There will be omissions due to assumed knowledge, omissions due to limited examples, miscommunication due to teacher-student knowledge differences, etc, etc. You may watch a lecture, or read a book, and think "I get it!", but you'll only discover all these gaps and misunderstandings once you try to actually apply yourself.

2) The decision making involved in acting will be based on what's in YOUR head, not that of the teacher. The words in the book reflect what was in the teacher's head, but when you are hands-on trying to solve a problem it's YOU that has to make decisions, and how the teacher may have perceived the same problem and tackled it is largely unknown and irrelevant to you. You need to learn to bridge the gap between the transmitted knowledge and your own thought processes when trying to apply it.

OutOfHere2• 2 years ago

And DOING can absolutely be fun. At least in his tweet, Karpathy seems to be missing the experience of DOING altogether. DOING should not be by rote practice, but by diverse interesting projects that apply what is being learned.

auselen• 2 years ago

I remember this being discussed: https://news.ycombinator.com/item?id=41914724

pchangr• 2 years ago

I think Karpathy is not in line with latest research. I’ll quote verbatim the following paragraph:

“Motivation plays a crucial role in facilitating LTM retention. It may derive from external incentives (i.e., reward and punishment) or/and intrinsic incentives (i.e., personal curiosity) [29]. During the learning process, students, driven by external incentives such as high scores in exams, are more likely to focus only on unfamiliar items to “recognize” them in exams, whereas items “recognized” but not yet “rehearsed” are skimmed. As a result, these items may never be recalled outside of testing, resulting in limited access to LTM storage [23,29]. On the contrary, students with high intrinsic motivation might find studying more effortful and thus develop personal insights into the content. Therefore, spaced repetition may indirectly influence LTM storage strength through the intermediary of intrinsic motivation. Specifically, it elevates students’ learning capacity, self-efficacy, and incentive for self-growth, while controlling for their baseline proficiency [29]. The underlying reason is that spaced repetition helps students maintain the conscious effort to continue engaging with the spaced learning schedule.”

You can find the original references in the quoted paper below:

Yuan X. Evidence of the Spacing Effect and Influences on Perceptions of Learning and Science Curricula. Cureus. 2022 Jan 13;14(1):e21201. doi: 10.7759/cureus.21201. PMID: 35047318; PMCID: PMC8759977.

Andrej Karpathy on learning (should never be fun)

Related Stories