Top Test Scores From Shanghai Stun Educators

http://www.nytimes.com/2010/12/07/education/07education.html?hp

by ghshephard • 16 years ago

41 45 16 years ago

ghshephard• 16 years ago

A nice description of the OECD PISA: http://www.oecd.org/about/0,3347,en_2649_35845621_1_1_1_1_1,...

This is a pretty decent summary:

"PISA is a three-yearly survey of 15-year-olds in the 30 OECD member countries and 35 partner countries . It assesses the extent to which students near the end of compulsory education have acquired the knowledge and skills essential in everyday life. They are tested in the domains of reading, mathematical and scientific literacy and complete a background questionnaire. For each cycle one domain is focused on more than the others. The first data collection took place in 2000, the second in 2003 and the third in 2006. The fourth cycle is in progress for 2009."

Like any statistical analysis, this study isn't perfect, but, it certainly is an eye opener. If China is able to replicate their success in Shanghai in other regions of China, the United States is going to start to fall behind in 5-10 years in terms of academic leadership of the world.

Time will tell - Shanghai might be an outlier. The United States may also get its act together when it comes to education in general.

ars• 16 years ago

This is not politically correct to say, but given the gap between white and black scores, I wonder how the US would do if you ranked it based only on white scores.

Also, when judging how well a country will do in science you should not look at the average, but rather at the top. Only a small number of people will choose a career in the sciences, so as long as you have those people, the ones at the bottom don't matter for this purpose - they bring down the average, but don't prevent a country from being a leader in research and development.

maxklein2• 16 years ago

How well would Germany do if it took away Turkish scores, and how well would France do if it took away Arab scores?

Perhaps the U.S would do even better if only Asian-Americans were allowed to participate.

St-Clock• 16 years ago

This is not politically correct to say, but given the gap in revenues between white and black, I wonder how the US would do if they had better redistribution of wealth.

nopinsight• 16 years ago

Your point about science is roughly correct but cannot be taken too far. A country's average will indirectly influence the R&D activities of the top group as well. Democratic support, educational funding, social perception all influence science career to an extent. For example, it is so much easier to rest on your laurel when everyone around you does much worse. And when the average performance of a group is raised even moderately, the number of people who passes a certain threshold increase substantially (assuming a distribution not too far from Gaussian).

Also, there needs to be a critical mass of good researchers to create a rich ecosystem that in turn produces significant technology. The US lead in R&D is partially caused by its large number of active researchers and the collective amount of funding these scientists receive compared to other countries.

zephyrfalcon• 16 years ago

Not entirely unrelated:

In some of these countries, school is mandatory for 15-year-olds (e.g. the Netherlands). In other countries, like the US, you can drop out by (or before) that time. I wonder if this skews the results in any way. In the NL, a random sample of 15-year-old students is essentially a sample of all 15-year-olds. In the US, it's a sample of all 15-year-olds who are still in school; in other words, you are not testing the results of kids who have dropped out. If you did, it seems to me that this would likely bring the overall score down.

brown9-2• 16 years ago

...given the gap between white and black scores, I wonder how the US would do if you ranked it based only on white scores.

Also, when judging how well a country will do in science you should not look at the average, but rather at the top.

What would be the point of such exclusions and comparisons?

I don't think that these surveys are merely meant to find out which countries are leading in certain subjects. Of course you can look at it that way, but I think a better way to look at it is to see it as a measure of how well each country's education system is doing at educating its students on average.

I think it's great if our top students are at the top worldwide in science, but I'm still really, really concerned that the US education system is failing to educate the average student.

RickHull• 16 years ago

> This is not politically correct to say, but given the gap between white and black scores, I wonder how the US would do if you ranked it based only on white scores.

Questions of political correctness aside, this seems like some fuzzy thinking to me. Why focus on skin pigment as a distinction? Presumably because these numbers are easily available, and the categories are relatively "neat".

But why not skip all that and just use the top quartile scores? Or the top percentile?

The relentless focus on skin pigment is damaging to our thought processes as a whole, globally. It's an easy target, and it is definitely a (relatively) reliable proxy indicator for certain attributes (e.g. income, test scores). But most interesting questions (and answers) are causative in nature, and with skin color only being a proxy indicator, it seems limited to correlative questions. Also, in our civilization's more primitive and less-informed times, skin pigment was a critical distinction in both daily life and foreign policy. However, this was simply due to the lack of better or easier-to-use distinctions.

I mean, does anyone truly believe that skin pigment causes lower test scores? I doubt it. Now, do inattentive parenting and poor schooling cause lower test scores? I think so.

It may be that skin pigment correlates with poor education at home and institutionally, and it may be the only reasonable "metric" for quality of upbringing. But the focus should be on the causative force, and shortcuts that we make for practicality's sake should be broadly disclaimed at the outset.

So let's drop our vestigial attachment to skin pigment as an explanatory or causative condition. Not for reasons of sensitivity or political correctness, but for reasons of epistemological correctness.

Also: most of the "issues" that people have with skin pigment are actually judgments of culture and not appearance. Again, appearance is merely a proxy for culture, here. Note that I see nothing wrong with condemning cultures that are judged to promote or result in "bad outcomes".

kahirsch• 16 years ago

For the U.S., there are breakdowns by race available. The detailed stats are only available on one of the big categories at a time (math in 2003, science in 2006, reading in 2009).

For 2003, the math scores by race are here: http://www.nces.ed.gov/surveys/PISA/PISA2003HighlightsFigure...

In 2003, students in the United States averaged 483 on the math literacy test and 477 on the problem solving test (24th place in the OECD in each). Non-Hispanic White students in the U.S. averaged 512 points on math literacy and 506 points on problem solving. That's equivalent to 15th place and 13th place, respectively.

NonEUCitizen• 16 years ago

A fairer comparison (city to city, with education-obsessed parents), would be between Shanghai and Cupertino.

jacobolus• 16 years ago

Really? Shanghai has 19 million people. Cupertino has 50 thousand.

jimboyoungblood3• 16 years ago

Not a fair comparison at all. Cupertino would easily win.

tedesign• 16 years ago

high probability of government organized cheating it's China and national pride justifies everything

laduch• 16 years ago

It's certainly possible, but saying it without evidence is an extremely weak argument.

riobard• 16 years ago

You know little about the Chinese education system.

All Chinese kids (I was one of them) are basically trained for these kinds of standard tests. For the most part of their school life, they are judged by the scores they get. The whole point of the system is not to educate, but to test, and based on scores, pick the few at the top for better schools to go next year, and the process repeats. This pattern starts from primary schools all up to graduate schools. That's all freaking 16 years!

It is no surprise they score high in this one. They do not need to cheat. Of course if they want they can, but my point still holds: they are battle-tested “exam machines”.

The problem, though, is that the approach kills creativity and innovation.

Threat? Hell no. As long as the States keeps recruiting the best minds with scholarship and offers visa for them to stay working there, I don't see any chance the States will fall behind in smartness.

shrikant• 16 years ago

s/Chinese/Indian/g

Would still be true. Personal experience here.

edit: I mean, I'm not one of the 'smart' ones. Bog-average performer here, going by my test scores.

zwieback• 16 years ago

This was in the Atlantic recently, looks at only the "top" performers in different countries since it's so hard to compare diverse populations and school systems. Yay for MA.

http://www.theatlantic.com/magazine/archive/2010/12/your-chi...

johnohara2• 16 years ago

About 5,100 15-year-olds in Shanghai were chosen as a representative cross-section of students in that city.

Who did the "choosing" and what defines "representative."

We were all impressed with the Chinese Women's Gymnastics Team during the last olympics. Then it turned out many were barely of age to compete. And they were very careful about interacting with the cameras.

Were I the U.S. education chief, I'd be talking to the Fins and Hong Kong Chinese instead.

colanderman• 16 years ago

Indeed, we've recently seen how much Shanghai is not representative of China as a whole: http://blog.sokanu.com/200-countries-200-years-4-minutes (skip to near the end of the video if you haven't seen it already)

If the US chose only students from say, California and New England, I'm sure we'd be competitive with China as well.

pashields• 16 years ago

I agree about China being representative, but even the best performing state in the US (Massachusetts) still takes a beating from a country like Finland. See the Atlantic's "Your Child Left Behind" - http://www.theatlantic.com/national/archive/2010/11/your-chi...

jimboyoungblood3• 16 years ago

Why don't you read the entire article first?

The testing in Shanghai was carried out by an international contractor, working with Chinese authorities, and overseen by the Australian Council for Educational Research, a nonprofit testing group, said Andreas Schleicher, who directs the Organization for Economic Cooperation and Development’s international educational testing program.

Mark Schneider, a commissioner of the Department of Education’s research arm in the George W. Bush administration, who returned from an educational research visit to China on Friday, said he had been skeptical about some PISA results in the past. But Mr. Schneider said he considered the accuracy of these results to be unassailable.

“The technical side of this was well regulated, the sampling was O.K., and there was no evidence of cheating,” he said.

johnohara2• 16 years ago

I did.

Mr. Schneider, however, noted some factors that may have influenced the outcome.

For one thing, Shanghai is a huge migration hub within China. Students are supposed to return to their home provinces to attend high school, but the Shanghai authorities could increase scores by allowing stellar students to stay in the city, he said. And Shanghai students apparently were told the test was important for China’s image and thus were more motivated to do well, he said.

Shanghai pop.: 19,000,000+ Finland pop.: 5,500,000+ Hong Kong pop.: 7,000,000+

The population of Shanghai exceeds the total population of NY, LA, CHI, HOU, and PHX combined.

Which student sample would you say is more diverse?

silverlake• 16 years ago

Canada scores significantly higher than the US. Yet I don't see Canada beating the US at anything anytime soon.

Hexstream• 16 years ago

How about at humility?

Maascamp• 16 years ago

And hockey of course.

yantramanav• 16 years ago

Am I the only one missing India from the rankings?

kschua• 16 years ago

From what it appears in the website. India didn't participate in this

ernope• 16 years ago

By the very nature of Shanghai there is tons more selection going on than in any US state.

1. Incredibly high land prices 2. long history of smart/rich people moving to the city - if you fail, you go back to your hometown 3. legislation trying to push people out of the city (only overcome by smart/rich people) 4. delayed puberty compared to average white or black american 5. no cultural value assigned to being rebellious

What percent of the US students tested were non-native speakers? What percent had spent years of their education overseas?

Finally, I don't know why everyone always talks about average education levels. I would much rather have the best top 10% or 1% in any field living in my country than have the highest average. And that's why I still feel pretty good about the US.

suhlash• 16 years ago

These tests are not as meaningful because it is not country vs country. But countries vs citiy states. Hong Kong, and Singapore are basically cities not countries. If South Korea only tested Seoul instead of the whole country, they would have also looked better.

jasonjei3• 16 years ago

People wonder if these test scores in China are beyond the norm. But the bitter truth is that these scores are indicative of their Confucian training, not any independent thinking that a "standardized" test could evaluate.

The Chinese education should be noted for a strong emphasis on rote memorization and the regurgitation of lectures. Unlike most Western philosophies of education (e.g, Plato), the Chinese education system does not instill or encourage creativity. What you will find is that many students that have undergone a Confucian education system are extremely capable within their competent subjects, but they will have a difficult time coming up with creative solutions other than textbook solutions.

Confucianism (as I have wrote before) was adopted aggressively during the Qing dynasty, because it perpetuated the importance of respect to the state. It taught that one should respect the state, the teacher, and the parents, because if one did all three, they would naturally respect the power of the king. In short it became a way to brainwash. And predictably, most Chinese innovation during the Qing empire diminished.

You'll note that Asian countries have a difficult time creating software. A lot of software in China are clones of American counterparts. It's because Confucian thought has made it difficult for many in China to be creative.

I've personally tutored students from China, and many of them who excelled through the Chinese education system found the American university system extremely challenging. The greatest challenge for them was writing an essay. Many of their essays would read like semi-plagiarized versions of the history text, without a pervasive POV that stayed close to the middle without making a single claim. These students would often blame the essay-writing to their language abilities. I told them that even if these essays were written in Chinese, you still don't have an essay because you don't have a thesis, and the whole point of the essay was for you to take a side. And then they would ask me which was the correct side...

The point is that, yes, these test scores are very impressive. But I don't think other countries--in particular, America--are exactly boned. The one redeeming attribute is that we can have any idiot in this country think he's a genius, in some extreme cases, become a world leader, and that's the sort of mentality that will save us. Until China has a Google or Apple that can develop iOS or Android. But then again, China isn't known for having a college dropout culture, so it's hard to see if they will have a Facebook, Microsoft, or Oracle given that the cultural differences.

micrypt• 16 years ago

It might not be your intent but your comment does sound a tad dismissive. Alibaba, Baidu and DangDang are, amongst others, considerably successful businesses. Perhaps their similarities to Western contemporaries are partly due to the fact that we are all human in the end and happen to have similar needs. However, they are remarkably innovative in their own contexts.

jasonjei3• 16 years ago

The culture in China doesn't entirely foster it. I agree China is entirely capable of producing excellent software. But that doesn't change the culture. I've breathed it when I lived in China for several months. You'll notice that college students in China are always in groups, and there is a hive mentality to decision making. The self isn't as important as the group. Espirit de corps.

Jack Ma of Alibaba spent considerable time in the US, and enlisted US friends to develop Alibaba. Many Asians that have been through Western education become extremely creative. Not that I'm saying the Western/US education system is better (it has its problems, on the entirely opposite scale), but Western university culture helps bring people to their potential, while the university culture in Asian countries is a continuation of high school (hard to get into, extremely easy to get out).

I work regularly with Chinese companies, and if you were to go to an equipment manufacturer in China, you'd notice they practically use the same designs, block diagrams, software for the same product. It's all exactly the same. You might also recall that the mandatory filtering software used by the Chinese government was plagiarized. Or the Cisco firmware. IP has serious problems in China.

Japanese cellphones, for instance, were way ahead of the curve on the merits of specs. But that isn't true anymore. Japan, like other Asian countries, have a difficult time with creativity not because they're less talented, but because the culture doesn't help bring ideas to fruition.

ovi256• 16 years ago

The last paragraphs of the article directly refute your point:

“While that’s important, for me the real significance of these results is that they refute the commonly held hypothesis that China just produces rote learning.”

“Large fractions of these students demonstrate their ability to extrapolate from what they know and apply their knowledge very creatively in novel situations,” he said.

The Chinese generation that will create their Google, Facebook or whatever is just growing up, it seems.

jerf• 16 years ago

I'd like someone to explain to me how doing well on an international standardized test somehow refutes the rote learning claim.

I mean, yes, I am every bit as skeptical as I sound, but it is a serious question as well, if someone has a concrete answer. It's very easy to say that these test results prove that they are not just pulling things out of their memory banks, but that doesn't mean there's any actual logic or truth behind that claim, and I'm at a bit of a loss as to how to get from the results to the claim.

jasonjei3• 16 years ago

Mr. Schleicher says that while reviewing the test results. The LSATs, for instance, is a test that requires knowledge applied creatively, too, but it doesn't mean successful test-takers are able to produce Google or Facebook. In fact, successful test-taking implies conformity with the graded standard, and is intended to measure how students stack up against a defined standard.

Would you say the US-equivalent of Advanced Placement exams be a good measure of creativity? It requires creativity in solutions too--but the real question is, can you TEST creativity? Can you test the ability for people to see problems? A test, by nature, has already presented a problem. It doesn't test if they can see a problem and devise a solution.

With enough practice, you can answer every tested problem type on a "standardized" test. A Chinese student told me that when he was studying for the SATs he looked for every single problem type and that once he had considered every problem type, it was just a matter of applying the same strategy to the problem. The key is that since the test is "standardized," you can look for the standard types of problems.

I'm questioning the culture, I suppose. Not the people. The culture in China hasn't really changed.[1]

[1] An interesting tidbit: Mao observed the effects of Confucianism and how it stilted innovation in China and launched the "Cultural Revolution" as a means to deprecate Confucianism.

geebee• 16 years ago

One thing we should keep in mind when evaluating these scores is that there is a different cost/benefit calculation in different countries, especially for math scores.

For instance, take a look at the "best jobs sorted by high pay" (from cnn)

http://money.cnn.com/magazines/moneymag/bestjobs/2010/highpa...

There are some techy jobs on there that do require substantial amounts of math[1] (software engineering director, actuary, computer and information scientist). However, the extremely well paid health fields listed generally require only a single year of an easier track of college calculus, and lawyers don't have to take any math at all. In China, most top government officials are engineers or scientists, whereas in the US, they're more typically poly sci majors who wen to law school.

I think that top US students have rationally concluded that you need to get pretty good at math to have access to the top jobs and positions, but that after a certain point, there's a diminishing return and an opportunity cost (ie., time is better spent on other subjects, playing sports, developing leadership skills, and so forth). Of course, this is true of all nations, but I think that the rewards for going farther with math are higher in China (and perhaps France) than they are in the US.

[1] by "substantial math", I mean educational paths that require you take the full two years of math/hard science/engineering track calculus, rather than the slower paced, one year track often offered to premed or economics students.

Top Test Scores From Shanghai Stun Educators

Related Stories