The Crowd Performs at the 95th Percentile

http://trada.com/blog/2010/10/15/example-how-crowdsourcing-works/

by vrish88 • 16 years ago

6 3 16 years ago

Interesting. Does this mean group think isn't as bad as we thought?

_delirium• 16 years ago

It depends on the thing being estimated. This approach works great when the main problem is variance among estimates, not overall bias. In that case, aggregating a bunch of predictions reduces the variance, as the biased-low and biased-high individuals cancel each other out. Variance between individual guesses often follows something like a bell curve, so the average is just the center of the bell curve. If the curve was centered over the right location, then the average is better than the vast majority of individual predictions, which are off to one tail or another.

There are plenty of things where the average prediction is systematically off, though, like people's estimates of how likely they are to die of various causes. In that case, averaging predictions won't produce the right result.

vrish88• 16 years ago

It also depends upon the elements present in the group. Obviously, riots and the "mob mentality" are undesirable. However there are ways to minimize these behaviors. In James Surowiecki's book, The Wisdom of the Crowds, he identifies 4 elements that need to present in order to have a wise crowd:

[from wikipedia: http://en.wikipedia.org/wiki/The_Wisdom_of_Crowds]

1. Diversity of opinion - Each person should have private information even if it's just an eccentric interpretation of the known facts.

2. Independence - People's opinions aren't determined by the opinions of those around them.

3. Decentralization - People are able to specialize and draw on local knowledge.

4. Aggregation - Some mechanism exists for turning private judgments into a collective decision.

The Crowd Performs at the 95th Percentile

Related Stories