The Evaporative Cooling Effect in Online Communities

http://blog.bumblebeelabs.com/social-software-sundays-2-the-evaporative-cooling-effect/

by shalmanese • 16 years ago

163 35 16 years ago

blog.bumblebeelabs.com

zacharycohn3• 16 years ago

I've seen this happening in several communities, namely some of the online parkour forums. One in particular used to be a very tight nit group of people, but as Parkour grew, more and more people got on the forum. As those original "high quality" people left, I thought the next level of people "took over" there positions.

This kind of kept happening, but every time it got worse and worse. It wasn't until reading this article that I realized that they weren't "taking over" or "stepping up," but they were simply the next highest level.

I've seen it happen and tried to warn people who manage those communities of it.. usually with little success. Great article though, explains the concept extremely well.

shalmanese4• 16 years ago

Thanks for the kind words Zach :).

acabal• 16 years ago

I think it's one thing to see it happening, and a totally different thing to actually know how to fix it.

I run an online community and what this article describes has happened to me a few times already, even with a karma/reputation system in place. I know I must fix the problem, but I don't know how right now.

Does anyone have specific suggestions on how to broadly lessen this kind of problem?

shalmanese4• 16 years ago

I agree, it's not an easy thing to fix and there's no broad panceas or it would already be fixed. I tried to put in some very broad level fixes in the article but it's really something that needs to be figure out on a community by community level and many communities are intrinsically designed so that it can never be fixed.

Mz3• 16 years ago

You can start by finding out what your "best" people want from the community and try to find a reasonable means to ensure they get it. (At the risk of sounding like an egomaniac:) I have left a number of online communities and seen evidence that amount and quality of conversation went way, way down after I left. In almost every case, I could have been kept if people would have just engaged me in conversation without turning it into either "let's kick the crap out of Michele" or "let's fawn all over and idolize Michele". My need is for a social (and intellectual) outlet and being idolized and then attacked for it doesn't remotely meet that need. I need people I can talk to who will actually speak to me like I'm a human being -- like I make mistakes and don't know everything but don't deserve to have the living crap kicked out of me because I made some stupid mistake. Letting forum members pounce on little mistakes made by your best members and blow them all out of proportion and turn it into a fucking federal case is a great way to encourage them to leave.

Other people will want other things from what I want. And you will need to find some healthy means to balance meeting their needs with the needs of the forum. I have seen forums go completely to hell because it got all twisted out of shape to meet the need of one or more top members in a really unhealthy manner. So while trying to find out what your best people want, don't let it become their personal pond to piss in either. You want to foster an environment that is highly likely to consistently provide things that more than one top member is looking for without specifically making it about them as an individual. Don't prostitute the forum to them as an individual but shape the forum to be highly likely to provide X, Y or Z so that most of your current top members are likely to stay but fresh blood will also be attracted.

HTH.

zacharycohn3• 16 years ago

The first step to fixing it is to understand what is happening, though. The reason I couldn't fix it in my example was that the people who were in charge of the forum didn't understand or agree that this was the core problem.

Eliezer• 16 years ago

Besides the original evaporative cooling essay, see http://lesswrong.com/lw/c1/wellkept_gardens_die_by_pacifism/.

auxbuss2• 16 years ago

Now, that's much more worth reading.

dkarl• 16 years ago

That's an interesting essay and a good explanation of why it's toxic for any group to defend their idiots, the idiots whom they tend to find on their side of the issues. In politics, a lot of the ill will between liberals and conservatives stems from each side's reluctance to disavow the kooks and cranks on their own side. In a fragmented online community, members who themselves make high-quality contributions may resist quality control measures because they don't want their obnoxious idiot allies silenced. But that results in them having to put up with the obnoxious idiot allies of their rivals, which makes intelligent debate impossible and results in the high-quality contributors leaving anyway.

johnglasgow• 16 years ago

This article is very relevant to our company as we redesign our social product to better achieve the network effect and reduce this exact issue. Thanks for the great insights.

atari3• 16 years ago

Good post. Have been thinking a lot about influence and reputation lately, and the relative nature of both of them.

Products that only cater to the Quora early adopter crowd won't ever make it big, but ones that are able to siphon them off such that they feel like they're special might.

barrkel• 16 years ago

The same thing happened pretty quickly with stackoverflow.com. I wanted a way to search for questions by users who had high reputation, simply as a way of increasing the chances of finding interesting questions to answer. Unfortunately, it doesn't look like it'll get implemented. I've long since stopped visiting SO except for questions related to my areas of responsibility in the company I work for's products.

il• 16 years ago

Can't you use their API/data dump for this?

tokenadult• 16 years ago

From the article: "In Academia, high school students have to fight to become undergraduates. Undergraduates have to fight to become PhD candidates. PhD candidates have to fight to become adjuncts. Adjuncts have to fight to become tenured and tenured professors have to fight to become Dean. I can’t even think of a single online community that bears even the slightest resemblance to this sort of power structure."

Interesting point. Maybe having forms of recognition that are demonstrably difficult to achieve does help outstanding contributors feel more like staying.

atari3• 16 years ago

Indeed, and it may incentivize and motivate them to do things they might not otherwise have done within the community.

zacharycohn3• 16 years ago

Or alternatively, if there is some sort of power structure that is difficult to move through, people don't bother because there are other places for them to go that don't require as much effort.

atari3• 16 years ago

Right, you need some kind of happy medium (not too easy, not too difficult).

auxbuss2• 16 years ago

Why would anyone engaging in an online community want to partake of an elitist "power structure". Okay, other than power junkies whose world is embedded in this kind of delusional existence.

Most folk don't want that insidious role-playing in real life. Online they can just walk away. So that would be why it doesn't exist.

shalmanese4• 16 years ago

In the real world, we take these elitist power structures for granted. Corporations, Politics, Churches, PTAs, they all unthinkingly replicate a power structure as they grow.

What's interesting to me is that this same pattern appears not to apply to the web.

_delirium• 16 years ago

While I'm academic and more positive on it than some of the sentiment around here is, that actually seems like one of the biggest downsides of academia. It at least partly selects for careerists/game-players/CV-padders over experimenters/thinkers (doubly so for anyone whose goal is to become Dean). Lots of smart people end up burning out or going elsewhere, and many of the rest have a lot of their time taken up learning not-very-interesting things, like how to negotiate the web of prestige, publication, and funding agencies in their field. To the extent it works, I think it's despite all the hierarchy, not because of it.

jacobolus• 16 years ago

This is a good point. I generally like the “show us the code” method of social proof in open source communities much more than the “show us your certifications” method in academia. The problem of course is that the latter is much easier to scale up, because the certification stands in for a close and careful examination of each person... “Oh, he graduated from X school? he’s probably alright” is much faster than “Let me spend a few hours reading a major piece of work.”

It’s interesting to wonder whether how you’d go about building communities for doing science in a way that had tasks for inexpert but hard-working/bright newcomers to get started on, and allowed building reputation without all of the hassles of degrees and academic politics where much of the effort is misdirected. I wonder when/if some of the methods and norms of online communities will begin to supplant those of academia as it is today.

Interestingly, when it was smaller, science used to be much more accessible in this way. Once you were literate, you’d passed enough of a bar for serious people to treat you with some respect, and all kinds of advances were made by amateurs. So are the more open systems in computing just because of the youth of the field, or does online communication allow amateurism to scale further than it used to?

shalmanese4• 16 years ago

I overemphasized the fighting element in the system and neglected to mention that each of these classes have very specific roles in the ecosystem and that they mesh together in a (sometimes healthy) symbiotic way to form the social structure that is the university.

lionhearted• 16 years ago

> After a while, you start to subscribe to what I call the Groucho Marx rule. You stop attending any event which would have you as a participant.

I ideally like to be the stupidest and least informed person in any room that I'm in. At the very least, not the smartest. Continually looking for groups of people that are challenging and stimulating to be around means very fast learning and growth. But, I think most people prefer the opposite - they'd rather be the star, or at least mostly on par with everyone else.

Admittedly, it's hard. I had a mentor of mine invite me on a cruise. Now, that's the kind of frilly thing I'd never buy a ticket to on my own, but I like being around good people. So, I went - and man, it's not fun being gently corrected on how to hold a fork because I was out of touch with the etiquette. Hold the fork upside down? What?

Embarrassing, but you gotta learn it somewhere if you run in crowds that care about that sort of thing...

Don't get me wrong - I also like to help people, take time to answer questions, be available for people who reach out, give back, pay it forwards. But I think taking the attitude of trying to be the most humble, hard working, and least gifted person among a group of people means very fast absorbing of lessons, though admittedly at the expense of constant amounts of little mistakes and embarrassments.

Edit: I should add - I think thoughtful new members are necessary to prevent evaporative cooling - by introducing new points of view and asking questions that people haven't thought about in a long time. Normally two very successful members of a community won't have a discussion about the fundamentals, but might really enjoy each other's points when talking to a third person asking smart, good questions.

nck4222• 16 years ago

I think that's exactly what he's saying. The top leave because they realize they don't receive any benefit from interacting with the community, until finally the only people that're left are the ones that are unaware.

"Each layer of disappearances slowly reduces the average quality of the group until such a point that you reach the people who are so unskilled-and-unaware of it that they’re unable to tell that they’re part of a mediocre group."

spitfire• 16 years ago

http://www.danford.net/boyd/destruction.htm

A US fighter pilot formally proved this back in the 60's/70's. He eventually went on to form a general theory for social systems (militaries, businesses, etc) in dynamic environments.

The fundamental takeaway is without outside input, systems tend to increase their entropy leading to collapse. You MUST have external inputs into any system in order to keep it alive.

Mz3• 16 years ago

I place a high value on treating all people with respect and making information equally accessible to all, yet I am increasingly frustrated with the outcomes that seems to get me. I find that a lot of forums have serious problems rooted in the fact that one or two or a few people set the tone for the entire forum and most folks line up to agree with one side and vehemently disagree with the other. This does not allow for a free exchange of information and ideas, which is something I also value highly. I tend to stand against that clique-ish trend, which frequently puts me in hot water. Yet it seems to me that the fact that I don't have a big ego and don't want to be treated like top dog on some list drastically undermines my credibility....but, I also think that the appearance of popularity I once had on a few small lists was more facade than reality. Spin-off lists I started never attracted more than a few members and my websites have never attracted much traffic. I suspect that a vocal minority made me appear more popular than I was.

This article is much food for thought for issues I have long wrestled with and couldn't seem to get any good info on.

stretchwithme• 16 years ago

The key to interrupting this effect is to somehow give the most desirable users a lot of influence and/or special status, perhaps being in a position to decide whether other users are worthy of the same status.

cullenking• 16 years ago

I have been giving alot of thought towards this problem lately, and am beginning to lean towards the rather undemocratic side of things. Meaning, not all comments or story submissions on a site like HN are equal - some users consistently hit the ball out of the park and some consistently say things that should be be said. Karma systems based on upvotes obviously detect this, however upvotes only apply to a specific comment/article, and don't follow an actual user. How do you make that power user totally invested in your site? Just rely on points? Make them a moderator? Flag "elite" member posts with a different background color?

Perfect timing for this article, as I am about to launch a niche news aggregator as side project. Trying to decide how best to handle the reputation system...

ursablanco• 16 years ago

You wouldn't be the first to twig to the idea that democracy is not the way to achieve the best of something - but that's not its intent - its intent is to provide some degree of fairness. And even then, it's willing to trade fairness for everyone for merely a guarantee of fairness for a majority.

If you locate a reputation system that produces better quality or decently filters better quality, please brag about it, I'm sure you'd have a success on your hands.

Mz3• 16 years ago

Here are two rules that I took from a quality forum I once belonged to which have their merits:

  1) Speak from firsthand personal experience.
  2) Or cite your sources.
  (3) or both)

My view is that you really should not be aiming to put certain individuals on a pedestal. You should be aiming to foster an environment with a high level of quality of conversation. If you put too much emphasis on specific members, that only makes it more likely that it will kill your site when they leave. Having the right rules, practices and culture in place is harder to do but more resilient. I don't care how idealistic, humble and/or brilliant any individual is, when some place starts catering overly much to that person's ego, it gets to the point where no one can disagree with them without all hell breaking loose. This is not a good way to foster good conversation and at some point it will start driving away good people.

Even on Hacker News, the most highly voted comments tend to come over and over from the same people. I have no reason whatsoever to believe that those individuals are simply always brilliant every time they open their mouth. I have every reason to believe that the karma system encourages members to jump on the bandwagon and upvote certain people simply for being those people. Having been the person on the pedestal, I think there is a serious downside to being on that pedestal and I actively seek to avoid being placed there. The only thing I have found that works well and consistently as an antidote to being consistently attacked is to deflect public praise. The one and only thing I know for sure is that the more some people heaped public praise upon me, the more other people felt the need to kick the crap out of me, prove I did not know everything, and try desperately to bring me down a notch. Conversely, not publicly thanking people for saying nice things about me, turning a deaf ear to it, and actively asserting that I don't have all the answers has dramatically reduced the degree to which I end up on the receiving end of ugly assaults.

(I cringe when I see other people having public praise heaped upon them. The fall is highly likely to come sooner or later and most people don't really know how to deal effectively when their adoring fans suddenly turn into a rabid lynch mob, a la how Britney Spears seemed to just get crazier and crazier the more desperately she tried to get back her status as America's Darling.)

cma• 16 years ago

This is covered in Micromotives and Macrobehavior, in a section dealing with clubs/meetings/etc.

http://www.amazon.com/Micromotives-Macrobehavior-Lectures-Pu...

A similar book focused solely on online phenomena would rock.

bluethunder• 16 years ago

Very Interesting Stuff.

I can see the evaporative cooling happening in HN itself over the last year or so.

Also, puts in perspective the correlation between the success of Yahoo Answers and their reputation system.

rdl• 16 years ago

I think it is worth considering how this applies to Hacker News? Hacker News is somewhat niche, although not particularly, and does not have particularly arduous requirements for joining or participating. Aside from the leaderboard and some informal recognition of really stellar contributors (pg, patio11, ...), it's a fairly flat system without rewards for status.

Dove• 16 years ago

Not to worry. HN readers have high standards and wield their downvotes aggressively. I've never seen such a well-maintained community as this one.

bincat• 16 years ago

Another essay to read on the subject is "Attacked from Within" http://www.kuro5hin.org/story/2009/3/12/33338/3000 .

I think the main reason is that communities grow only so big and then they turn into a society. Users recognized in a community will lose their recognition that evaporates into the mediocre society.

The Evaporative Cooling Effect in Online Communities

Related Stories