Khan Academy: The Illusion of Understanding

http://theeducationscientist.blogspot.com/2014/02/khan-academy-and-illusion-of.html

by akbarnama • 12 years ago

43 65 12 years ago

theeducationscientist.blogspot.com

So it would overflow, right, dependent on how much of the cubes extend above the surface?

dredmorbius4• 12 years ago

What is Archimedes' principle? How does it relate to density? What is the association of density to volume? What happens to ice as it melts (or water as it freezes)?

webspekt• 12 years ago

The puzzle had me fooled twice (after 2 seconds: water level sinks, after the article: solution cant be determined), but thanks to your hint it finally dawned on me

Unosolo2• 12 years ago

Since water reaches its peak density at about 4 °C, the actual outcome will vary slightly depending on the initial and the final temperature of the water once the ice has melted.

nkoren2• 12 years ago

It's a trick question. The question creates doubt because it asks you to choose between two wrong options. [Edit: no, it gives all three possibilities; I speed-read through that. Thanks to others for pointing that out. However it then does predominantly focus on the question of "up or down?", which isn't the right question.]

Ice is lower density than water, which is why it floats. As the ice melts, the runoff from the above-surface ice will raise the water level -- but the shrinkage from the below-surface ice will lower the water level. Assuming the above-surface and below-surface ice melts at the same rate, the water level will remain unchanged.

(Note that if this were a glass of hot water in a cold room, then you'd get non-uniform heating of the ice. The subsurface ice would melt first, causing the level of the water to lower, until the above-surface ice melted, at which point the water level would return to its original level. On the other hand, a glass of cold water in a hot room would initially overflow while the above-surface ice melted, and then its water level would lower when the subsurface ice melted. So the answer to the original question is, depending on context, either "neither" or "both".)

kaybe• 12 years ago

The ice is floating, it would just lie higher or lower in the water depending on what melted first. No change of water level.

jessriedel• 12 years ago

The question creates doubt because it asks you to choose between two wrong options.

I think you misread the post:

Does it rise and over-flow the glass, remain constant throughout the melting process, or go down?

Also,

Assuming the above-surface and below-surface ice melts at the same rate

You don't have to assume anything about ice melting rates. All the info necessary to solve the problem is provided.

croddin• 12 years ago

It isn't a trick question. The article gives all three possibilities.

brador• 12 years ago

The question also completely ignores evaporation, which, since we're waiting for ice to melt, could be significant to the problem.

sudhirj2• 12 years ago

I think it would go down... here's why.

When water freezes, it expands. That's what full bottles in a freezer burst, as do pipes in winter. So 1 gram of water occupies less space in the glass (1 ml) than 1 gram of ice.

According to Archimedes' Principle, a floating object will displace precisely as much water as its weight. Which is why a log of wood floats - the water it displaces weighs more than the wood itself.

If 1 gram of ice is floating, it means it has already displaced 1 gram of water. When 1 gram of ice becomes 1 gram of water, it will occupy less space.

As the ice melts, the level of water comes down.

sudhirj2• 12 years ago

Or maybe the ice above exactly replaces the level of water, and on the whole it stays the same. The more I think about it the more confused I get, which seems to be the point of the article.

prezjordan• 12 years ago

But there is some ice peeking above the surface. For this reason, I feel like the level should stay the same as the ice melts.

My only concern is the ice being "held down" by other ice cubes. It doesn't get the same treatment as the ice peeking above the surface.

randallsquared• 12 years ago

Yes, it does. As long as it's not resting on the bottom, the bouyancy of the ice underneath the top ice will push a little more of the top ice out of the water. The amount of ice matters for how far out it sticks, but the number of pieces it's divided into doesn't matter.

squigs25• 12 years ago

condensate on the ice will add to the glass, making it impossible to answer this question without knowing the humidity and temperature.

james10712• 12 years ago

Suppose we remove an ice cube of mass 10g.

The ice cube was previously floating and was therefore displacing 10g of water.

If we let our ice cube melt, we will have 10g of water than we can add back to the glass.

The net result is that the water level remains the same.

nationcrafting• 12 years ago

Exactly.

I've had this discussion many times w.r.t. global warming, and am amazed so many people, including scientists, just don't get it. The level the seas would rise is not so much dependent on floating ice (north pole) since the displacement is exactly the same, but on ice that was on top of land (e.g. Greenland, Antarctica, etc.).

pja3• 12 years ago

Absolutely, although it's worth noting that loss of floating polar ice will change the albedo of the earth, so there's a positive feedback loop there that may result in further warming & consequent melting of land-based ice elsewhere.

So melting of the floating polar ice will not directly result in raised sea levels, but 2nd round effects of the melt might do so.

james10712• 12 years ago

The way to understand this, for me at least, is to think of the hole left by the ice cube and to fill it with water from the melted ice cube.

That makes it much more concrete.

DanBC2• 12 years ago

What about the densities of salt sea water and frshwater iceburgs?

lingben• 12 years ago

ok, water expands when it is frozen and contracts when it melts, so the question is by what factor to allow for the water to either overflow or not.

and are we talking about icecubes floating in the glass or filling up the glass so they are standing on each other (like a cube skyscraper) from the bottom of the glass?

spindritf2• 12 years ago

FTA:

When examining the ice you note that the cubes rise just above the surface of the water (like glaciers in the ocean), but do not extend to the bottom of the glass.

skywhopper• 12 years ago

The article is explicit in re your question: the cubes are floating, not stacked.

michaelochurch• 12 years ago

I find that I have the opposite problem. I think I know less than I actually know. Sometimes I feel (as a mathematician would) that I don't understand something unless I can prove it, assertion by assertion and line by line.

I dropped out of grad school after one year, and it seemed like a bigger risk/deal than it was. My sense is that the major benefit of having a PhD (especially from a top school) is the confidence that comes with it. It gives you faith that you know enough to tackle the interesting problems, and an assurance that people (usually uninformed people, often without the educational credentials they overvalue) won't try to peg you down as not up to it. Socially, it gives you the right to be an expert, which is not the same thing as expertise but just as valuable (if not moreso) in the work world.

So, even in traditional education, I'd argue that much of the benefit earned (maybe 85%) is confidence, both your own and others' in you. You can get a lot on your own by reading the papers and books, but you never get that official approval or that "proof" that you learned the right stuff.

Oh, and the answer is that the water level stays exactly the same. Archimedes' Principle is that a floating object displaces as much water as it weighs. When melted, the iceberg (90% submerged) takes up as much space in the water as the submerged part occupies.

ZeroGravitas3• 12 years ago

Luckily not a single student ever left traditional education with a single misconception about anything, or else this would seem like a ridiculously slanted article.

edit: on re-reading, he's still clearly using Khan and MOOCs as eye-catching buzzwords, symbolic of "bad education", but a) he mention (in passing) offline education having similar issues, and b) he intends to talk about how online education could fulfil his pet theory of learning in another article, so it's not quite the usual reactionary post, although it does lean that way.

jacobolus• 12 years ago

The point is, if you have some kind of human discussion, close examination of coursework, etc., it’s possible for a sensitive expert teacher to figure out what kind of misconceptions and misunderstandings students have, and talk through them, making sure that students leave a course with a correct model in their heads. By contrast, this is very difficult to achieve via lecture alone, whether in person or online. [And certainly many traditional classes fail at this.]

In particular, all of the Khan Academy lectures I have seen (admittedly, this is only about a dozen random ones) do quite a poor job compared to the good teachers and professors I’ve had. They don’t seem to me to be grounded in much understanding or appreciation of how people learn.

(In theory, online courses should give people access to enough interactive tools and communication infrastructure to discuss and question and properly figure out the gaps in each student’s understanding. In practice, I haven’t seen any which do this at all well.)

ZeroGravitas3• 12 years ago

And what about the people who will never have professors, and have never had sensitive expert teachers?

Do we just explain that they can't afford a good education and that it sucks to be them as the people who could afford that education are now fighting tooth and nail against any move to democratize learning?

mathattack• 12 years ago

It does use Khan Academy as a strawman though. "Let's say that Khan Academy is the be-all end-all of education, then prove that it's not."

In reality, sites like the Khan Academy aren't intended to replace classroom education. They're meant to reinforce it. Get the lectures on your own time, so that you can get to the deep learning with teachers in the classroom. Spend classroom time on the socratic method, rather than lectures.

He beats up the tool that he should be using to improve the classroom experience.

msandford• 12 years ago

Right? It seems like he's railing against the Kahn Academy but it's not as though a public school (or most private school) educations are much better. With a student/teacher ratio of 30:1 (or worse) and absolutely no tailoring of teaching method to student's optimal learning methods (verbal, visual, writing, etc) it's not as though traditional schooling is a paragon of getting everything 100% right and MOOCs are terrible by comparison.

Kahn should get a lot of credit for giving students the option to go back and re-watch things they didn't get the first time. If they learn by hearing or seeing (instead of reading) then that's an incredibly valuable tool.

If we want to complain about anything I think we should be complaining that there's no standardized testing of learning methods and then informing the students of their results (and what those results mean) so that they have the opportunity to try and optimize their learning and thus hopefully get a damn diploma. Why isn't that a national goal? Seems like it would be a small incremental change that would make a huge difference in terms of raising education levels across the board. It helps smart people and dumb people and everyone in between.

jerf4• 12 years ago

The reason I'm confident that in the long term MOOCs will succeed where traditional education fails is not that MOOCs will come out of the gate a superior product. The reason is that MOOCs can be tuned and developed over time in a way that a classroom experience can not, because every class room experience is too unique, delivered live, unrecorded, etc. It will take time for this to happen, but it is likely inevitable, and eventually the idea of flinging out an "education" into a room, where it shall disappear forever with no feedback or development, will be considered as archaic as the horse buggy.

(And also people can't seem to help comparing the perfect conventional education that does not exist to the worst of MOOCs, which may feel good, but is not relevant.)

VLM4• 12 years ago

I watched video tapes in the early 80s and before that, professionally produced film strips. Even film strips with 33RPM vinyl records, before tape. Also I saw a couple really old analog film movies in my youth. Currently, youtube videos and DVDs are supposedly popular. Why didn't they replace traditional classroom education using the same criteria you're applying to MOOCs?

You may be correct about outcome, but the reaction mechanism is lacking.

I'm a classically self educated well rounded kind of guy, so I feel education is developed from within not a mechanical external process much like weightlifting. The idea it merely requires butts in seats, and the result depends on what technique someone else applies to those seated butts, is more likely to disappear. Or at absolute most, perhaps "necessary but not sufficient".

yread• 12 years ago

WTF people!? Why is everybody discussing what the water is going to do when the article is about online education?

To stay on topic: I agree with the article, watching videos often leaves you with feeling of understanding but being unable to answer questions which require actual comprehension

mattgreenrocks• 12 years ago

Why is everybody discussing what the water is going to do when the article is about online education?

It's easier to argue about 'right' rather than the harder issue of deep understanding as it relates to MOOCs.

crankyguy• 12 years ago

I agree, I used to watch tons of nature shows and was always amazed at how I could recall very few of the facts presented in the show once the show was over. Retention doesn't occur until you have actually used the information in some way, not just passively watching it. I like the approach ULearniversity takes where you watch a short video and have to apply that technique before seeing the next video.

wbillingsley2• 12 years ago

At one of the keynotes for the Association for Learning Technology back in 2007, the speaker had a variation on this question. His point, if I recall, was a little different --

As normally asked, the question is code for "parrot the Archimedes principle back to me". But on a realistic "hot summers day" there's not just the Archimedes principle. There's evaporation that can remove water from the cup. There's wind, and occasional movement of the ice cubes as they melt and slip over each other in the glass, that could cause spillage if it's really that full that it's on the verge of overflowing. As could interruption of the surface tension (e.g., condensation in just the wrong spot at the edge of the glass, causing the water in the meniscus to spill over.) The initial temperature of the water being unstated, its density will also change a very small amount, potentially in either direction.

Often when we ask our students scenario questions as teachers, we are essentially asking them to identify which taught principle we have flagged this question as being about, and base their answer on that, ignoring all other factors.

If the student answers "A. It goes down", did they fail to grasp the Archimedes principle, or are they a pedant grumbling that over that length of time there will be evaporation so it won't be exactly the same?

Take the question out of the context of a course, and suddenly the educational point of it can change. Is the person asking you this a physics teacher checking your understanding of Archimedes principle, or an ALT conference pedant checking whether you understand that in uncontrolled environments where bumps and spillages can occur you can't blindly expect the ideal result you'd get in a textbook?

The peril comes when questions are used outside of a teaching context. For example in technical interviews, where the same question can sometimes be a signal for many different teaching points, each expecting subtly different answers, and the interviewee does not know which the interviewer has in mind.

dredmorbius4• 12 years ago

But on a realistic "hot summers day"...

That's an excellent point and a flaw of many "reasoned conclusions" thought experiments. There's a world of difference between contemplating what might happen (which is a useful first step) and experimenting to see what actually does happen.

In the case of your hot summer's day instance, there are a number of things which could confound the experiment: how do you distinguish between water inside the glass (from the initial water + ice), and that on the outside (mostly from condensation, but possibly also from spillage)? How could you control for this? How might you run the experiment repeatedly? Etc. Actually, a chemistry lab where students each mix known weights of water + ice at specified temperatures and observe what happens, then collaboratively compare results might make for a useful group + parallel experiment exercise.

And that's just for ice water. In a field I've got some interest in (practical and applied economics) there's an even greater tendency for assumptions and ideal circumstances to get constructed.

globalpanic• 12 years ago

Interesting and challenging article about most current online education offerings. It would be interesting to see if the problems of maintaining engagement in a MOOC were at least partly due to students realizing that some or all of the requirements for authentic understanding were not being met.

As I understand it, there is a different MOOC philosophy that has not gained much support (probably because it's much harder to scale) - the connectionist approach. Perhaps that would support some of the requirements.

(PS: interesting that many of the comments here try to rise to the ice-cube challenge...)

matchu• 12 years ago

I disagree with the article's assertion that, because I'd forgotten Archimedes' principle, I never truly understood it. I'd successfully worked the problem far enough to realize that my missing piece of information was how much of the ice cube stayed below water, then I watched the video and reacquired that information. No problems here, and the KA video was plenty sufficient for my needs.

pja3• 12 years ago

As the responses to this HN post demonstrate so clearly, an intuitive understand of what Archimedes Principle means in the real world is not something that is easy to acquire: I'm sure most of the commentators on this HN post are intelligent people who probably did well at school, yet the majority of the responses here are completely wrongheaded.

If it isn't transparently obvious that the water level will neither rise nor fall but stay exactly the same then you don't really understand Archimedes Principle.

The author's question therefore is: how is the Khan academy approach supposed to improve on this situation? A bare explanation of Archimedes Principle clearly isn't enough, nor is the ability to parrot said explanation after viewing the explanation it as many times as required for the viewer to learn the required words. How then is a learner going to acquire authentic understanding from Khan academy videos?

(Note that traditional education has clearly failed here too! But that's a side issue: the question here is why should we expect Khan academy to do any better?)

himangshuj• 12 years ago

can't agree more. I have been studying education sector in India, home to largest number of potential beneficiaries of khan academy. I would say khan academy is already bettering normal education. most teachers here would have no clue of archimedis principle atleast khan academy is giving the right information whether all the students are able to absorb is another matter

jcampbell1• 12 years ago

If it isn't transparently obvious that the water level will neither rise nor fall but stay exactly the same then you don't really understand Archimedes Principle.

Surely it will overflow once the ice melts. At about 4C the water will reach it's lowest point, and then rise and start overflowing at about 8C.

pja3• 12 years ago

The density change in the liquid water due to heating / cooling over the likely temperature range is tiny compared to the result of going from ice to liquid water. But I'll award you pedantry points nonetheless :)

The precise outcome requires knowing the volumes and shapes of the ice and water components, their initial temperatures and the temperature of the ambient environment so that you can calculate the path of the water volume over time as the ice has melts. In order to make that calculation you'd still have to make use of Archimedes Principle in order to know that you didn't need to worry about how much water was displaced by the floating ice in your calculations, which was the original point.

crazy1van• 12 years ago

Khan might not be perfect, but it is still pretty high quality. The lessons are certainly higher quality than most of what I encountered in school.

VLM4• 12 years ago

"If you answered “Yes” to this last question, then you just experienced the Illusion of Understanding first-hand."

Bzzt wrong it could mean you're thinking like an engineer instead of a scientist. And its not binary, so fuzziness would be partially one or the other reason at the same time.

Possible engineering challenges all contributing a really small delta, all of which add up to so few sig figs a good engineer can't predict what will happen to the water level:

1) You don't know how well mixed the cup is. I assure you, especially on the "summer day" in the article you can get a spread of 20 or more degrees in that cup. So the liq at the bottom is already way past the max density temp and expanding while the top still has some ice.

2) You don't know the starting temp of the water and ice. In addition to #1 above, the ice in my freezer is well below 32 and who knows the starting temp of the water. Maybe its barely melted meltwater or maybe its out of the tap or maybe the coffee maker. So its going to go thru some wild temp and density swings based on unknown initial conditions.

3) The pix shows massive condensation on the glass AKA they're running the experiment in New Orleans not Phoenix also in written form they're running it in the summer. Realize that there's nothing magic about surfaces and condensation... if a sq inch of 32F surface condenses 1 teaspoon per 5 minutes, the top will condense just as well as the sides. On the other hand if you run this in an environment with a dew point below 32F like where I am today, then water will steadily evaporate away depending on the intensity of energy striking the surface, although not too fast.

4) Ice stacks up nicely and clogs pipes. So you'd need more like a slushy to make sure that icecubes are not mechanically sticking up above the surface by being jammed by the cup walls. The supplied pix strongly indicates the middle ice cube is mechanically jammed way above flotation level. One thing for sure, liquids don't have much shear strength and thats going to eventually melt and drip down and raise the water level... how much, who knows.

5) How does water surface tension vary WRT temp? I donno. Probably does, a little. So how much the level can go up or down before it drips is a mystery. You might be able to tolerate a rise of 0.1% if the surface tension enables a "lens" above the top of 0.2% because of increased temp. Or maybe it goes the other way and surface tension drops so "just barely overflowing" at 32F means a flood (well, at least a drop...) at a liquid temp of 40F.

Its probably possible to very carefully engineer the perturbing contributions to a low enough, or counteracting against each other enough, level (oh the pun) such that the science principle will be demonstrated. Or given way the heck more engineering data, a properly engineered answer might be provided, maybe with enough sig figs to mean something. Or maybe not enough sig figs to mean anything. I'm feeling the latter.

brudgers• 12 years ago

The article conflates two separate connotations of "better" which are commonly applied in educational contexts. When we talk about better teachers like the author and Mr. Kahn himself, "better" refers to the average effectiveness of the teacher's methods on individual students. We would for example control for class size and socio-economic status of the students when deciding whether the author was a better teacher than Mr. Kahn himself.

But what "better" often means in terms of educational infrastructure is more access. We would measure the effectiveness by looking at the institution's effect on an entire population. Here we can look for the effects of access and raw numbers matter. A MOOC course could produce a significantly lower average achievement among individual students but produce orders of magnitude more highly successful outcomes.

It appears likely that they do based on my anecdotal experience. If there are 100x teachers and again I believe that there are in the context of adult education, then MOOC's can be better in both senses. Mr. Kahn may not be a 100x teacher, but he is a 100x educator.

Dan Grossman of University of Washington has taught "Programming Languages" on Coursera. I took the second section. Here: http://www.youtube.com/watch?v=1T4IQrOJr5U he talks about data he collected from the first session. Particularly interesting are what "number of students" and "passing" mean.

tokenadult• 12 years ago

The guest author is Professor of Education at the University of Texas at Arlington. It's good that educational research continues around the world. For the moment, it's especially important that education research focuses on what happens to young pupils in conventional classrooms, because they far outnumber older students who use MOOCs to learn. In that respect, I think the title "Khan Academy: The Illusion of Understanding" is unfair, because there are plenty of learners with illusions of understanding leaving the elementary schools of my country and yours, even though those learners have hands-on experiences in the classroom.

Michael Shayer in Britain[1] has been doing great research for years about how early experiences of children shape their understanding of the world. He has discovered that many of Piaget's findings about how children proceed through developmental stages in understanding no longer replicate in current children, because children today have different kinds of early life experiences from children in Piaget's era. What happens as society changes over time is that some experiences that used to be commonplace become rare, and other experiences that used to be rare become the most influential experiences in children's development.

The best way to gain intuition about Archimedes' principle (the example given in the blog post kindly submitted here) is probably to do the experiment repeatedly with differing experimental conditions and careful observation. The history of physics around the world shows that for centuries great thinkers could be badly confused about very elementary principles of physics. Galileo did the world a favor by insisting that physicists do more experiments to check their hypotheses. Teaching elementary pupils to hold their conclusions about the world tentatively and with an attitude of testing out everything they believe is the best kind of elementary teaching. Whether by video or in person, good teaching encourages learners to draw on their life experience between lesson sessions to test out ideas and to see what ideas are really correctly understood.

[1] https://kclpure.kcl.ac.uk/portal/en/persons/michael-shayer(5...

arikrak• 12 years ago

Schools find it easier to get students to memorize facts and techniques instead of getting them to really think and understand. This is a particular problem with Lecture-based learning, and it applies not just to middle-school students, but to Harvard students too:

http://harvardmagazine.com/2012/03/twilight-of-the-lecture

quasque• 12 years ago

The author's paper on the topic (http://sloanconsortium.org/sites/default/files/6-Schwartz_0....) makes for interesting reading as it expands on the five requirements he states are necessary for authentic understanding.

Though having read it, the only conclusion I can make is that I don't really understand anything about the subject.

alphaBetaGamma• 12 years ago

I think we don't understand anything about how to teach.

Formal education started at least 4000 years ago (earliest writing is at least 4,000 year old, and I think you need a formal education to have scribes who can read and write). I don't think we have done any substantial progress in how to teach in that time frame, at least compared to how much progress we have done in most other subjects: medicine, math, engineering, metallurgy, you name it. heck, we are still discussing what methodology we should use to teach reading & writing, and we have had 4000 years to figure what's the best way.

On the other hand, anyone who has had a good teacher can attests that teaching quality can vary by a huge amount. So my theory for good education is: 1) find good teachers (which probably includes "pay them well") 2) let them do their stuff

Anybody who tries to do something better than the above should be able to explain why their pet theory has not been discovered before.

DanBC2• 12 years ago

Don't let them "do their stuff". Have double blind randomised controlled studies (or as close as you can get) to test various teaching methods. The. Implement the good methods.

This has not been done because people have a built in desire for "fairness" and splitting a country or state into two groups means one group is getting a worse education. (Even though we don't know before the experiment starts which group that is)

chris_wot• 12 years ago

Whenever I read these pieces, I always think that the establishment is getting worried. Whilst there are good points to be made, such inflammatory and sensational headlines make me think "Now why are they so upset about this?" - and unfortunately I often feel that it's because they feel threatened someone is walking on their patch.

bhattisatish• 12 years ago

Eric Mazur's Peer learning process https://www.youtube.com/watch?feature=player_detailpage&v=Ww... kind of highlights the problem and a solution for the same. Namely, Flip the classroom. Ask questions that highlight the knowledge gap within the students. Let students debate among themselves and then actually provide the solution.

Though this is not sufficient, it at least improves the ability of the student to reason about the problem and question the models they have built around their heads on the given subject.

Khan Academy: The Illusion of Understanding

Related Stories