The Dollhouse Mafia, or "Don't Display Negative Karma"

http://buildingreputation.com/writings/2009/10/the_dollhouse_mafia_or_why_to.html

by rms • 17 years ago

66 31 17 years ago

There can't be general purpose reputation online. It's meaningless. There can only be reputation for specific behavior. Someone who has a lot of positive reputation on Ebay is reasonably likely to fulfill your order, assuming your is similar to the previously filled orders - check if all his/her other orders were for $1 and your's is for a million.

Just about any other karma system winds up measuring how hyper-active someone is online. My karma score tells you nothing about me at all except that I'm here a lot and talkative.

This can be good thing. Much real life bad or good reputation is unjustified.

rms2• 17 years ago

I don't know that it's going to happen anytime soon, but I can see exchanges between karma and real currency eventually leading to a reputational economy. See Doctorow's Down and Out in the Magic Kingdom. http://craphound.com/down/download.php I wish that pg would start allowing karma transfers here so we could start experimenting and I could cash in all of my stupid points, but that experiment is not one of his goals.

For a while I've told myself that I'd like to auction this username for charity, mostly just to see what happens, though I like the idea of converting my years of activity here into a charitable donation.

On Hacker News, karma really does just measure activity. My point is that it is theoretically possible to give karma more meaning than it has today.

unalone• 17 years ago

Wait, how would karma transfers work? What would be the point?

bhousel• 17 years ago

My point is that it is theoretically possible to give karma more meaning than it has today.

... or less, if they were to implement a karma trading system.

Bluem00• 17 years ago

You might be interested in the book "Accelerando" < http://www.amazon.com/exec/obidos/ASIN/0441014151/charlieswe... >, as one of its plot points is that the basic unit of exchange in the economy changes from money to reputation.

cojadate3• 17 years ago

I mostly agree with you, but it's not hard to create a karma system that isn't just a reward for hyperactivity. For example, Hacker News could change its system so that nobody got a point simply for posting a comment.

But however fine-tuned the system, no point system is immune from gaming, and no-one can expect points to be any more than vague clue as to someone's true worth as a participant in the community.

bd• 17 years ago

"For example, Hacker News could change its system so that nobody got a point simply for posting a comment."

You don't get a point simply for posting a comment: all points come from upvotes - downvotes. Initial "1 point" label is a purely cosmetic thing.

mey• 17 years ago

Karma/Reputation systems on website are to reward a desired behavior. Part of HN's value is it's active useful conversation (vs /.,digg etc). So it makes sense to give the community the tools to reward what they want and create positive self feedback. (Also like the other reply, simply posting doesn't give you karma)

Perceval4• 17 years ago

I agree that voting systems on the web end up favoring the most persistent, rather than the most worthy or highest quality. This is something that I've seen on Wikipedia and Digg. And to an extent with the trolls on Kuro5hin.

I think moderation is important, but that it should not be the central aspect of user interaction.

alttab2• 17 years ago

The article outlines only one major reason for karma-based systems.

People can get addicted to their rating, creating "karma whores" and the like. This is nothing that probably hasn't been discussed here and on other communities.

Its a balance between the web-site operators wanting people to come back (what is my karma now after that awesome comment?), and pissing people off due to the problems karma systems introduce along with a general decline in community quality (Reddit anyone?).

Is it too much to ask for people to just have online conversations for the sake of getting involved? The only real CLEAN motivation is intrinsic, but that says a lot more about human nature than online rating systems.

Perceval4• 17 years ago

I agree, which is why I support downplaying the egoistic elements of online communities. Hiding usernames where possible, hiding raw karma scores or post ratings, etc. The less systemic/administrative information plastered all over the place, the more the users will (hopefully) focus on content.

jcl• 17 years ago

On the other hand, I think the HN karma system has worked well to maintain focus in the site; lame attempts at humor get smacked hard, as do "+1" posts, coarse language, ad hominem arguments, etc. If the post rating were hidden, you wouldn't know if a down-voted post was due to one person disagreeing with you or a major breach of etiquette.

(Of course, in several ways, karma is a poor motivator for behavior that is valuable to the community.)

camccann• 17 years ago

People can get addicted to their rating, creating "karma whores" and the like.

A counterpoint to this: If a karma system reliably rewards beneficial behavior, "karma whores" can potentially create a lot of real value for more casual members of the community (at the cost of possibly some irritation from active but not addicted members). This may not be possible in all communities, of course, but here's at least one example: http://stackoverflow.com/users/22656/jon-skeet

nazgulnarsil• 17 years ago

you've just reinvented the invisible hand concept.

for the record I agree completely. checks and balances on negative behaviors is stupid and has never worked. what you need to do is set up the incentives so that everyone's goals are complementary anyway. then the users police themselves because everyone is benefiting. you've turned a zero-sum game (status) into positive sum.

blasdel• 17 years ago

The problem is that "karma whores" gravitate towards actions that are merely structurally beneficial without regard to actual value. It's cargo-cult commenting.

anigbrowl• 17 years ago

Everyone has a different idea of what karma should be for. Conspiracy theorists trust other conspiracy theorists and loathe anyone with a mundane explanation. Bicyclists like each other more than car owners.

So what you need is to give everyone their own karma table to rate other users. You arrive, and you are neutral towards everyone else. After a while, you vote some people up and some people down. As you establish what your own standards are, you are establishing your own weighting of different social behaviors. Your weightings are likely to correlate strongly with those of other users - most likely the majority group, but maybe you have a lot in common with an established minority crowd (eg UFO 'believers'). New encounters can then be assigned provisional karma scores based on the average scores handed out by the group with whom your own weightings correlate most strongly. Thus, if you are a UFO believer you will see provisional negative karma for the science geek who loves to debunk UFO theories, while if you are yourself a science geek you'll see provisional negative karma for UFO believers in accordance with the fact that your karma votes correlate strongly with similarly-minded people.

Characterizations of UFO believers, scientists etc. are based on internet stereotypes, not responsible for lost ego or damaged feelings

Obviously, a system like this with N users has N^2 karma scores and the cost of administering the karma system increases exponentially compared to linear user growth. But there are a lot of algorithms you could use to flatten this, since in practice there will be groups whose membership follows a power-law distribution, rather than purely atomistic noisy preferences. Another possibility is a local (client-side) karma table with p2p lookup.

Another downside is that groups will tend toward partisanship since there will not be any universal penalties for most unpopular views (obviously, some views are so extreme that they'll never gain any social traction); even minority thinkers will have a circle of friends who share an unpopular belief and provide mutual support (as can be seen across the internet). But this system accommodate pluralities, whereas attempts at universal karma result either in relative homogeneity (vast #s of echo chamber websites) or else the complete breakdown and abandonment of the karma concept, with attendant signal degradation (eg 4chan/b/).

eru• 17 years ago

> Obviously, a system like this with N users has N^2 karma scores and the cost of administering the karma system increases exponentially compared to linear user growth.

Do you imply that N^2 is exponential? Or is there another way to understand your sentence.

The rest of your post is quite insightful.

jlees2• 17 years ago

One way around the mafia problem is a kind of normalisation, though it has its own flaws. A negative vote from someone who always gives negative votes counts for less than a negative vote from someone who generally gives positive votes. You can add in thresholding and voting ring detection for some fun, resulting in normalisation of their opinion to some degree.

There's also normalisation on the other end, how many of the total votes are negative or positive - but this doesn't really work unless there's a transaction cost, as fake accounts can shill the voting.

Perceval4• 17 years ago

One of my ideas about moderation and rating systems is that the act of moderation tells us more about the person doing the moderating than it does about the story/post/person being moderated.

If we think about moderation as relational rather than objective or absolute or aggregating, we can treat the user's moderation activity as their particular web of relations to various other objects (stories/posts/users).

We could then drop all these relationships into an algorithm built to examine large matrices of relationships, say... PageRank. I wrote a tl;dr piece on moderation and online communities here: http://www.kuro5hin.org/story/2009/3/12/33338/3000

jlees2• 17 years ago

Funny, I've also been doing some thinking about moderation and implicit/explicit voting recently. Fascinating topic. Here's my piece: http://www.trendpreneur.com/online/link-voting-real-time-res...

abecedarius• 17 years ago

Advogato's trust metric fleshed out this sort of idea. I wonder why it (and related work) hasn't seen more use.

raintrees• 17 years ago

What about using a small graph instead of a sum total to show karma earning/loss over time?

chrischen• 17 years ago

Personally I think it would be better to only allow upvote, and reserve flagging for inappropriate comments. I don't feel it's necessary to downvote anything except if it's racist, offensive, or in any other ways inappropriate.

I mean I saw a person who was obviously not an English speaker post a hard to understand comment. He was voted down a couple of times but I can't understand why. His comment wasn't offensive. He may have had some misunderstanding and some trouble communicating but I don't think he had any negative intent. It seems in these cases negative voting serves no purpose. If you just want good comments highlighted, upvoting should be enough. Flagging can be used for innapropriate stuff.

The Dollhouse Mafia, or "Don't Display Negative Karma"

Related Stories