Common-knowledge effect: A bias in team decision making

https://www.nngroup.com/articles/common-knowledge-effect/

by vimes656 • 3 years ago

163 57 3 years ago

taneq• 3 years ago

“A person is smart. People are dumb, panicky animals and you know it.” - Agent Kay, MIB

nvy• 3 years ago

It's common to see this effect manifest during the Mt Everest simulation[0], which is sort of a touchstone in business school organizational behavior classes.

[0] https://hbsp.harvard.edu/product/8867-HTM-ENG

sp332• 3 years ago

Paywalled.

gonzo41• 3 years ago

Maybe the lesson is. Let green boots go. You need to get to the top.

zeroonetwothree• 3 years ago

It seems like some teams were able to reach the correct outcome. I wonder what they did differently.

growingkittens4• 3 years ago

Familiarity is a huge factor in decision-making.

Apple, a multi-billion dollar company, made the Apple Watch to track your health. Something was missing for half of the population: cycle tracking. No decision maker was familiar enough with women's health to understand the importance of aligning cycle information with other health information.

sltkr2• 3 years ago

People keep repeating this story, but is there any real evidence to support that this was actually an oversight, and not a deliberate decision?

Apple has a long history of making decisions that seem to go against what users want, like removing the headphone jack, and making a wireless mouse that can't be used while charging. Both of these decisions were made deliberately and intentionally.

For the headphone jack, it was about establishing Apple products as high-end products for fashionable people who can afford to buy accessories like AirPods. Unfashionable people use outdated black Android phones with cheap wired headphones. Removing functionality that is primarily used by unfashionably users is a brand decision, not an oversight.

The story about the Apple mouse is really the same: if Apple allows the mouse to be used while charging, lazy people will leave it on the cable most of the time, and then it looks indistinguishable from a $5 USB mouse. That's not good for Apple's brand.

So finally back to Apple Health and the Apple Watch: I can easily imagine that Apple omitted period tracking intentionally to position these products as aimed at young professionals who take their health and fitness seriously (a fashionable group), rather than "menstruating women” (an unfashionable group).

growingkittens4• 3 years ago

I can easily imagine that Apple omitted period tracking intentionally to position these products as aimed at young professionals who take their health and fitness seriously (a fashionable group), rather than "menstruating women” (an unfashionable group).

Maybe Apple should put out a statement saying that "We did this on purpose because professionals don't have to worry about menstruation." That would clear things right up.

chongli• 3 years ago

For the headphone jack, it was about establishing Apple products as high-end products for fashionable people who can afford to buy accessories like AirPods

That might have been one of the goals but I doubt it was the only goal. Another reason for removing the headphone jack is dust and moisture ingress protection. It's difficult to achieve an IP67 rating in a phone with a 3.5mm TRS jack. This is due to the shape of the connector and the extra thickness you need to add to accommodate a jack with sealing rings. By switching to the lightning connector exclusively, Apple was able to achieve IP67 ratings in all their phones after the change.

This is a huge improvement for users because previously a lot of phones were being brought in for service showing liquid damage. Having your phone die due to dropping it in some water is a terrible experience that is now quite rare due to the IP67/IP68 ratings of modern phones.

egman_ekki• 3 years ago

I can easily imagine that Apple omitted period tracking intentionally to position these products as aimed at young professionals who take their health and fitness seriously, […] not menstruating women.

Yup, like for example Mikaela Shiffrin who doesn’t take her fitness seriously [1]. Or Wimbledon tennis players [2]. Not like those young professionals…

1. https://time.com/6279881/periods-sports-gender-bias/

2. https://www.forbes.com/sites/kimelsesser/2023/07/05/wimbledo...

slim• 3 years ago

did not see any of the large companies include such a simple and obviously useful app. my guess is it's too much responsibility and lawyers are against it

robertlagrant2• 3 years ago

Something was missing for half of the population: cycle tracking

Can a watch detect this? What does that add vs an app that a cycle is input into?

growingkittens4• 3 years ago

The watch doesn't necessarily detect cycles directly. Cycles can influence things a fitness watch traditionally tracks. There was no option for input in the OS.

donutshop• 3 years ago

This smells like "Industry best practices"

asimpletune• 3 years ago

I think this might be one reason why pair programming can be difficult at times. I’ve personally found success in pair programming, but it usually came after adopting an attitude of moving past the stuff we know and trying a more free, humble approach.

This isn’t the best way to put it, but for lack of better words just starting with the mentality of admitting that it’s ok not to know what to do next and that we’re here to solve it really helps fight the kind of bias the article talks about.

It’s when you get to the point that you’re comfortable just reasoning aloud that the beauty of pair programming comes to light. Silly mistakes happen constantly but they’re caught right away and you move on so quickly in your shared state of excitement that a lot of the inhibition just goes out the window.

Mileage varies though, as not everyone gets the same benefits that I described.

wpietri• 3 years ago

Yeah, I think of this as the locker room effect. If you're not used to it, undressing in a locker room can feel scary. You're naked! But once you get used to it, you realize everybody is naked and it's fine.

But that's conditional on everybody not being jerks. Anybody who tries to score points by pointing out weaknesses is toxic to open and honest collaboration.

oooyay• 3 years ago

Scary I think is one thing, and I certainly think some people feel exactly this. Other folks I think struggle with maintaining this very direct but neutral voice that works well in pair programming. One that reflects curiosity, scrupulousness, and empathy all in one voice. Others I think struggle with the idea that it's more difficult to control the image people see when they're so rawly exposed to people.

geraldhh2• 3 years ago

Anybody who tries to score points by pointing out weaknesses is toxic to open and honest collaboration.

probably well intended, but no.

phreack• 3 years ago

My best experiences with pair programming have been with people that I trust a great deal, enough to know that we'll definitely still be cool after arguing code. I don't think it'd be of any use at all unless we were both comfortable enough with each other to be intellectually honest, and able to both criticize our opinions and agree with them when appropriate.

Some people are afraid of conflict and would rather agree when they really don't, some other people are just looking to "win" or always poke holes at everything but their own takes, both types aren't very useful in the end.

In my experience, when you find that sweet spot with someone who complements you, pairing up is just the best, definitely more than the sum of our parts.

otoburb• 3 years ago

>Deliberating in person isn't superior to online. According to research by Simon Lam and John Schaubroeck, virtual teams are more likely to overcome the common-knowledge effect compared to face-to-face teams, perhaps due to easier access to notes and materials.

That was an interesting observation, although the study didn't mention that another confounder could have been non-verbal communication often being much more suggestive/stronger (both to reinforce or suppress influence) in-person.

smcin• 3 years ago

Yes and the section "Team Decision Making" is really unclear about how exactly "Afterward, team members got together and were given 10 minutes to share, discuss, and determine the best project" happened:

- were each participant's rank, credentials, job title, age, experience visible ("7% Execs, 4% Prod/PM, 4% Marketing, 44% Designers , 23% UX Researchers, etc.")? Were the participants or their ranks known to each other beforehand?

- was it a roundtable? huddle? debate? unstructured free-for-all? egalitarian? squabble? shouting-match? Did they take a show-of-hands (or voice-vote) at the start and/or end to find out who chose which option?

- how exactly did the discussion dynamic go, which people or considerations dominated it, how did people decide who to let speak or listen to? Did people defer to the perceived expert/ highest ranking person/ perceived spokesperson for the majority opinion/ loudest person, or not? There can be multiple cultural and nonverbal factors. Were the Designers (or UX Researchers) more influenced by their peers than other people? Did participants make statements like "in my X years experience as Y, criterion Z is important"?

- I think the researchers (Nielsen Norman) should have run multiple controls where the same group of people get the same statements, but then vary both the "discussion" format and participants (e.g. different subsets of 7 from 9) to see how much that influences the outcome. And then do it on Slack and/or Zoom (both text-only, voice-conf and video-conf) and measure the influence of that too. Or even old-school (asynchronous email): limit the "discussion" to each participant can write one group email to everyone else, then they have 10min to individually consider their decision. And also measure whether it's influenced by whether they're anonymous, or sign their full name, or include job title.

civilized• 3 years ago

Is that a confounder? I think that's an essential difference between the online and in-person experience.

cma• 3 years ago

Unless smell and touch are important it can be done with VR. Probably not until Apple's is out with finally enough clarity for coding, though apparently still somewhat bad ergonomics.

otteromkram• 3 years ago

People could have online meetings with cameras active in order to catch those nonverbal cues.

Didn't we just have some of the highest productive ever during COVID?

BinRoo• 3 years ago

This study is comparing individuals having complete knowledge vs. a team with individuals having incomplete knowledge. Yes, the communication complexity tax must be paid. What am I missing?

abdulhaq• 3 years ago

you're missing what the summary says: teams often make worse decisions than individuals by relying too much on widely understood data while disregarding information possessed by only a few individuals

staybailey• 3 years ago

I think OP is on to something here. In particular the study design makes it hard to tease out issues specific to common knowledge verses anchoring or bias towards ones own knowledge. Notably each individual's information set makes B look worse than A or C.

I think the study would have been more compelling if the common knowledge favored A and C but each individual's total knowledge was neutral between the A,B and C. If results favored not B in that set up it would indicate that folks were specifically anchoring on the common knowledge itself rather than anchoring on their initial hypothesis.

pixl97• 3 years ago

This is a group effect quite often seen in other places.

Tornado chasing is a good example. Take an experienced tornado chaser near a storm, and they will commonly back off at a safe distance when things are looking sketchy.

But if you take that same chaser and put them in a group of people of varying skill levels they can get complacent and stay in a dangerous area much too long. Because the low skilled people are commonly worse at identifying dangerous situations, they don't begin to worry/panic in situations they should. The higher skilled people will commonly ignore their own feelings in that case, probably because of some kind of innate human group dynamics, where we see those people being call and think they are misjudging the situation.

dkural2• 3 years ago

It's not that the other people have incomplete knowledge, it is that they disregard it.

vanous• 3 years ago

It's not that the other people have incomplete knowledge, it is that they disregard it.

From the article:

"If each team member is provided with all project statements, then the team can match the decision-making effectiveness of an individual (which is about 80%)."

So depending on who got which portion of which information is the critical part here, bringing communication and social skills into it. Human skills are then much more important then pure technical knowledge. This seems to be the truth in other settings as well.

dash23• 3 years ago

You are missing the following statement: "If each team member is provided with all project statements, then the team can match the decision-making effectiveness of an individual (which is about 80%)."

I.e., they are comparing privately-informed teams with publicly-informed teams, not with individuals. They just simplified the exposition.

chiefalchemist• 3 years ago

If you're going for "none os us is smarter than all of us", the flaw in such group decisions is that they are made openly in the group. The first opinion influences the next and so on, and the louder and most persistent opinion gain more weight in the final decision than they deserve. The biases pile up and the quality of the decision goes down.

In order to take advantage of the knowledge of the many, each individual needs to form their opinion independent of the influence of the others. From there you move on to a structured and mediated discussion (i.e., not an adhoc free for all). Of course, participants can change their minds, but they do so based more careful considerations and far less based on the emotions and biases of a traditional group decisions.

See "The Influencial Mind" by Tali Sharot for more details.

https://m.youtube.com/watch?v=2HMsEVmnhZE

notarobot1232• 3 years ago

Project statements were distributed unevenly

This seems to be comparing the effects of the distribution of mostly positive information about two bad projects with the distribution of mostly negative information about one good project.

The source of the stated bias cannot be concluded without isolating for the effects of the other variables and this study seems to be lacking significant permutations of information sentiment, distribution strategy and project quality to be meaningful.

It could also be the case that negative information spreads more easily or that positive information is harder to introduce into a group than negative information. Both of these conclusions seem equally derivable from the results of this very limited study.

dash23• 3 years ago

Both of those would indeed be possible mechanisms. But the point is that an efficient team would be able to share its members' private knowledge. After all, they all have the same goal, and they all know that all the information is (by assumption) true.

It's probably true that you'd expect private knowledge not to be 100% shared in the real world. But the effect size is surprisingly large, especially in such a simplified setting. Literally, the teams in the private info treatment could have just said "let's all write what we know down on paper and share it", and they'd have been in the same boat as the other treatment!

notarobot1232• 3 years ago

Participants were given a disproportionate number of positive statements for projects A and C so each participant may be biased towards a more positive interpretation of the somewhat ambiguous statements (sufficiently ambiguous to cause 20% of individuals to make different judgements without that initial bias).

Even if the information was interpreted consistently by each individual, the benchmark of success is still biased against collaborative decision-making. The possibility that the collective judgment is different, and perhaps better, is precluded by the assumption that the group should reach the same judgements as individuals.

If the group produced more balanced or well-informed judgements due to the distribution of expertise, would that be interpreted here as a failure of information efficiency and "decision-making quality"?

owenpalmer• 3 years ago

Does anyone actually believe that teams make better decisions than individuals?

dartos• 3 years ago

Yes.

dash23• 3 years ago

These guys do: https://wires.onlinelibrary.wiley.com/doi/abs/10.1002/wcs.11...

"Researchers have compared the strategic behavior of groups and individuals in many games: prisoner's dilemma, dictator, ultimatum, trust, centipede and principal–agent games, among others. Our review suggests that results are quite consistent in revealing that group decisions are closer to the game-theoretic assumption of rationality than individual decisions."

Do you believe that there is a single right answer to your question, so that teams always make better decisions than individuals, or alternatively always make worse ones?

fendy3002• 3 years ago

Depends on situation and conditions. In this UX case, assuming that information is ready, a team of 10 UX experts with similar expertise and have been working together for some time will reach the good decision.

A team consist of CEOs, managers, devs, UX, marketing/sales & finance won't get the same level of decision assuming the information is ready. However they'll be useful for information / idea gathering / brainstorming when the information is not ready, for a single person / team to make decision.

My past experience sums up to this:

* Team of similar expertise & knowledge: good & fast on decision making, lack on seeking knowledge in other part of expertise

* Team with variety of expertise & knowledge: will make slow & bad decisions more than good or fast one, in exchange of decent chance of knowledge sharing for each expertise

Of course the quality of each member, how can they work as a team, whether they have hidden personal agenda / hating each other plays part in the quality and speed of decision making too

adra• 3 years ago

I'd say teams are way more likely to avoid clearly bad decisions.

Now your statement is so clearly absolutist in its tone, so to speak specifically, I would say teams will largely point to the best outcome eventually but often pays the bill in over lost efficiency.

Individuals make bad takes, sometimes they have personality issues, sometimes then have a terrible day and are dialing it in at work who knows. What I know is that it's less likely that an entire team are living their worst day at the same time which could lead to catastrophically bad decisions.

ranting-moth• 3 years ago

There is a grain of truth in it when they say that the IQ of a team equals the IQ of the most intelligent team member divided by the number of people in the team.

Aloha• 3 years ago

I have a similar 'rule of crowds' - for any group over three, their collective intelligence drops by 1/3rd for every person added.

PredictorX1• 3 years ago

Someone once said that "A hundred fools do not make one wise man."

AlbertCory• 3 years ago

It's ancient news that group decision making only helps if the group members don't influence each other.

See "The Wisdom of Crowds"

esafak• 3 years ago

Deliberation is bad? That doesn't sound right. I think it could only hold true in limited contexts.

hyperpape• 3 years ago

While interesting, their study has a weird relation to the headline claim. It’s not unrelated, but it’s not a demonstration of the claim (I don’t see that they measured the time teams spent discussing each statement), nor does it really a seem like a consequence of the claim.

So while I appreciate some of the reminders about decision-making it’s an oddly structured article.

skybrian2• 3 years ago

Assuming this is an effect we care about, it seems useful to researchers to have a simplified model of it. How much it generalizes is hard to say, though, if this is the first we’ve heard of it. Maybe someone familiar with the literature could say more?

skybrian2• 3 years ago

It seems like a rather artificial scenario due to the available information being explicitly written down? And yet they reproduced the effect. That’s pretty interesting.

Would it still happen if people were told what the experiment was about and they came up with a strategy first? One strategy might be to copy everything they got to their notes and combine them at the beginning of the meeting.

I wonder if there’s a lesson in that for real life.

Common-knowledge effect: A bias in team decision making

Related Stories