Does Diversity Trump Ability? An Example of the Misuse of Mathematics [pdf]

http://www.ams.org/notices/201409/rnoti-p1024.pdf

by todd8 • 12 years ago

111 53 12 years ago

As someone who knows it intimately, I would like to say that this kind of thing is endemic across the social science literature. What is frustrating is that only very few people seem to have the ability to understand why it is not sound.

jev2• 12 years ago

Probably because the mathematicians don't seem to have the ability to explain why it is not sound without devolving into obscure symbols and jargon.

amathstudent3• 12 years ago

I think that's a very important point.

Indeed, when you try to explain something simply to people, they often won't believe that it's really that simple, since there's all those symbols!

dxbydt3• 12 years ago

If this was html & written to the point instead of stuffed in a pdf & written up in this roundabout manner, it would be on top of HN with 100+ votes :) I read the whole pdf but here's the TL:DR; anyways -

Randomization improves algorithms. This is so obvious to CS folks its taught in basic cs101 - the ones i'm familiar with are where you pick a random pivot element for quicksort, or say the one where you draw a circle inside a square & pick n random points inside the square, four times the number of points inside the circle will equal pi as n becomes large, or you resort to the miller rabin test for primality when you are doing those rsa calculator type problems where you pick keys for encryption, which will randomly sample some number of possible witnesses and call the number prime if none turn out to be witnesses, or using monte-carlo methods to compute integrals for functions for which no closed-form formula exists etc....there's like tons of examples where you introduce a little bit of randomness & it'll speed up your algorithm.

So this Page+Hong wrote a paper where they want you to randomly hire employees ( who they call problem-solvers or agents ), because diversity trumps ability ?! So if you have 1000 applicants, instead of hiring based on some metric like ivy-league/test-scores/IQ/github/whatever, you just hire randomly, because the randomness introduces diversity which trumps ability ?!! To prove this nonsensical point, they introduce an artificial math problem where agents proceeding randomly obtain the right answer, & not doing so gets you stuck. Ergo, diversity > ability. Sheesh.

revelation• 12 years ago

Have you read the linked paper and the one its discussing?

I find the critique vastly more readable.

kitsune_• 12 years ago

Because I didn't have any money after high school I decided to get a degree where I could work part time. I had been programming for quite some years but had an interest in media and politics. So I started to study media sciences at the largest university here in Switzerland. After a month I realized that I had made a huge mistake. There was a class called 'statistics in social sciences' and the assistants had trouble with basic stuff like calculating the mean, median, avg etc. They even celebrated their mathematical illiteracy to a certain degree. On top of that, much of the writing in the standard text books could be reduced to meaningless tautologies once you deconstructed the complex phrases a bit. I dropped out very quickly after that. The whole experience left a very sour taste in my mouth.

nostromo• 12 years ago

I was shocked to read the original article. I can't believe that such overarching conclusions about human nature were drawn from a little toy box computational experiment. You might as well publish a psychology paper after a weekend playing The Sims.

http://vserver1.cscs.lsa.umich.edu/~spage/pnas.pdf

kenjackson2• 12 years ago

It also seems to defy common sense. How many top ACM programming teams randomly pick programmers from the college? How many great bball teams are randomly picked from players? There are few activities, where excellence is easily measured, where randomly picking participants results in superior results.

johnbender• 12 years ago

It's important to realize that common sense is never a replacement for a proof or emperical result. That is, common sense is not sufficient to refute the claims made by the original paper.

The power of science is that it often defies reasoning and in so doing provides new insights into how things work.

DanBC• 12 years ago

Moneyball shows us that all that knowledge and experience used to pick a baseball team was wrong and that people should ignore their "common sense" (which is usually just another word for "cognitive bias") and should use hard data.

In the absence of hard data just picking a team randomly froma pool of qualifying players seems to me like it would be as good as pickping players based on the team-picker's experience of the game and "cultural fit" and other bullshit.

HelloMcFly• 12 years ago

If anyone is interested in better research on diversity and team outcomes, I recommend looking into research on "faultlines" in teams. It is an interesting way of operationalizing diversity and predicting its effects.

Link to abstract: http://amr.aom.org/content/23/2/325.short

I can't find a direct link to the PDF at the moment. That's the original article. There's been a lot of research since testing the theory, typically supporting it (though I'm not 100% up to date on this topic).

bayesianhorse3• 12 years ago

In finance it has long been known that diversity in a portfolio can often trump the performance/ability in a particular stock.

Turns out, you can see portfolio theory popping up outside of financial markets, not only in assembling a team, but also advertising campaigns, ant colonies and bacterial colonies.

And yes, I know that the linked article debunks a paper that abuses math... It's just that portfolio theory is at least an analogy to understand why diversity can trump ability. Within reason.

Guthur• 12 years ago

In my opinion the reason "portfolio theory" works in the cases that you mentioned is that it most perform well across a time series within a highly dynamic environment. But if one is allow to pick an optimal team for each discrete problem then it will surely outperform.

In my mind diversity will help when you have little to no a priori knowledge and then for can not predict what attributes you will need.

bayesianhorse3• 12 years ago

Have you never seen a software development project which ran into unanticipated problems?

What about "knowledge work" is not a highly dynamic environment?

ohsnap• 12 years ago

Portfolio theory does not work here. Totally different concepts. MPT is about minimizing risk - not trumping the performance of a stock.

bayesianhorse3• 12 years ago

MPT is not about "minimizing the risk". It's about minimizing risk for a desired reward. In an abstract way of speaking, selecting team members is very similar.

roguecoder2• 12 years ago

Also the empirical studies from MIT about group problem solving. They have observed that diversity contributes more than the abilities of the individuals. It's not that the idea that diversity trumps ability in an unpredictable situation is wrong: it's that the original mathematical argument doesn't demonstrate that.

cjdrake• 12 years ago

Richard Feynman has some relevant thoughts on this subject: https://www.youtube.com/watch?v=IaO69CF5mbY

nether2• 12 years ago

Page’s work on diversity has been cited by NASA, the US Geological Survey, and Lawrence Berkeley Labs, among many others

Oops.

nether2• 12 years ago

One of the authors, Scott Page, majored in math at the University of Michigan. He also teaches the course "Model Thinking" on Coursera: https://www.coursera.org/course/modelthinking. Perhaps one to avoid...

dxbydt3• 12 years ago

Hey that's a great course. Pls do not avoid. Highly recommend signing up. The reason he wrote this famous paper & the book that evolved from that paper, in his own words - http://vserver1.cscs.lsa.umich.edu/~spage/thedifference_inte...

He says - "progress and innovation may depend less on lone thinkers with enormous IQs than on diverse people working together"

Not too sure about that. At all.Especially not in math, the subject he majored in.

eropple• 12 years ago

He says - "progress and innovation may depend less on lone thinkers with enormous IQs than on diverse people working together"
Not too sure about that.

Why would that not be a thing?

I mean, I am not the smartest person on my team. The list of things I know very little about is long. But I occasionally ask questions that are really obvious to me that reorient the discussion because people didn't think of them--problems that were Too Obvious To See up close, you know? And, similarly, when working in stuff that I do know a lot about, I find myself ignoring things that are to me so obvious and basic that my brain just goes right by them.

Innovation isn't like Civilization, you aren't just generating lightbulbs. Diversity of thought process leads to some inefficiencies, but it helps you reach new global maxima.

Russell912• 12 years ago

Am I missing something or is the author of this pdf's proof equally flawed. They assume each function is idempotent and injective ... meaning that it is the identity function. And the proof doesn't follow. A much more natural way to have "fixed" the original proof would be to require that V: X -> R be injective.

4bpp3• 12 years ago

Assuming by function you refer to the "agents'" \Phi, where do they make the assumption that it is injective? I see nothing to the effect in the text, and the functions in the counterexample on page 6 are not injective.

Russell912• 12 years ago

Oh, oops. I was reading their statement: This interpretation is incorrect without the additional hypothesis that V (x) is a one-to-one function. as - all of the phi functions must be one-to-one. Looks like they made the same assumption that I would have.

This was bothering me though so thanks for your response.

4bpp3• 12 years ago

The response paper seems to harp a lot on how the circumstance that typically, N_1 and N >> k, precludes any realistic applicability of the original theorem -- but it admits a fairly self-consistent and altogether much more sinister interpretation if you call the different \Phi "groups" (or "races" for maximum creepiness). The assumptions then say something to the effect of "every group's approach to problem solving produces fundamentally predictable results, and no group can be the best at solving each type of problem", which is a meme that in some form has been floating around in diversity activism circles for a long time.

The "mathematical theorem" then simply captures the triviality that under this sort of worldview, you want your team to contain Zorblaxians (sorry, SMBC) because there are some Zorblaxian problems that Zorblaxians have a natural affinity towards, and nobody without such an affinity could possibly make progress on. Since this is not stated explicitly, the statement becomes invokable even in settings in which otherwise a large number of people would raise eyebrows at the smell of exoticist quackery that the idea of "different ways of knowing" exudes.

QuantumChaos• 12 years ago

Your post captures the problem much better than the linked article.

I would summarize the problem as being that the result is tautological, but the gloss of mathematics gives it the appearance of not being tautological.

If people said "diverse teams are better because people from different backgrounds are good at solving different kids of problems" then that would be accurate, but leave room for debate about whether then assumption is true. If they say "diverse teams are mathematically proven to be better", this would be inaccurate, but give great ammunition to argue that science supports progressive views.

im3w1l• 12 years ago

Has there been any attempt at quantifying the "wishful thinking bias" in social science?

amathstudent3• 12 years ago

Here's something you might find interesting: http://vserver1.cscs.lsa.umich.edu/~crshalizi/weblog/698.htm...

There have are also loads of replication projects, some currently ongoing, of both experimental and statistical papers, with generally dismal results.

rlvesco7• 12 years ago

There is one positive aspect to all this that I would like to point out.

Most social science does not use math, so it makes it hard to argue against ill-defined concepts and ideas. So the fact that these mistakes were found is actually a positive thing and shows one reason why math is useful. That's not to say math cannot be used to obscure arguments and assumptions, it can, but words can do that too.

pervycreeper• 12 years ago

That was surprisingly entertaining. Any recommendations for other academic papers in that vein?

Does Diversity Trump Ability? An Example of the Misuse of Mathematics [pdf]

Related Stories