When sites like Digg and Reddit revert to the mean -- is HN next?

http://www.michaelalanmiller.com/?p=1996

by quoderat • 17 years ago

24 55 17 years ago

Of course it's next. It's just a matter of time. This is obviously the natural life cycle of internet forums, and has been since the beginning.

Story quality is directly linked to user growth. When you have a small niche site with "niche-expert" users, you get a high interest and visible actions; as the site grows, individual actions matter less, the niche is expanded into those that may only have a casual interest in the topic, and the forum inevitability changes to meet the needs of the average user - which is not the same type of user that previously existed. As the more windows are broken, the more "decay" is introduced into the forum. But the site is doing exactly what it's supposed to do - mirror the average demographic. It's that demographic that changes.

You simply cannot grow a website like this and maintain a niche focus. So if you choose to welcome more and more people, this is the end result. The sites that have been successful at maintaining focus control either the story submission (/.) or the entry of users (MeFi).

Sites like HN and reddit need to limit growth the same way a small startup shouldn't focus on growth per se if they want to retain that particular environment.

How to do this successfully is another problem altogether.

pg2• 17 years ago

Of course it's next. It's just a matter of time.

One can always claim this about something that hasn't happened yet. But it's already been two years and 15x growth and the sky has not fallen. So you could just as easily argue that what has saved us so far (killing fluff stories, insisting on civility in comment threads) will continue to work.

run4yourlives7• 17 years ago

I'm not suggesting that the "sky will fall", I'm suggesting that it is inevitable that this site will change if it continues to grow, user count wise. To suggest it hasn't evolved and changed from it's original state is disingenuous really. (Not that you are doing that, specifically)

Digg and reddit have turned into a glorified dorm room because that's what their average user is, a male college kid. Clearly, many people do feel the user base of this site changing - we wouldn't be commenting on this thread were this not the case.

It's not a good/bad thing, as it means you've chosen to broaden your base as opposed to remain niche. There's nothing wrong with that. But at some point, you hit the magical land where the average user demographic has changed so much that the tipping point is passed.

icey• 17 years ago

Philosophically HN is different than reddit and Digg in that the person who runs the site is active in shaping the ongoing conversation. Reddit and Digg ultimately became for-profit companies and in doing so, embraced the democratic method of policing their sites.

HN on the other hand is somewhere between an oligarchy and meritocracy; and can only work that way because pg doesn't appear to want to monetize it.

My point is that reddit and Digg are far more similar than reddit and HN or Digg and HN.

motoko• 17 years ago

news.ycombinator is successful because it's first the web forum for YCombinator program participants. "Trimming" works (civility and killing fluff stories) because what remains is a community of vetted YCombinator members and their friends.

gruseom• 17 years ago

When I took freshman philosophy, our professor told us about induction ("it's always been this way, so it will continue to") and anti-induction ("it hasn't been this way yet, so it probaby will soon"). He then explained that the argument for induction was that it's always worked before, so it probably will in the future, and the argument for anti-induction was that since it's never worked in the past, it's bound to this time. :)

whughes• 17 years ago

Perhaps the best method to fix this is to leave it alone and let the 'free market' work things out. Another site to succeed the current generation is due to pop up soon. It will probably try some new things, and if they work then it will have a longer golden age. Gradual evolution could patch these issues. There also seems to be room for more than one site (Slashdot still exists, after all), so this process can go on in several places simultaneously.

makaimc• 17 years ago

It's very interesting to me that so many other people had the same experience with Digg and Reddit that I did. At first, I read it obsessively but now I can barely stand to scroll through the headlines. HN has gone through a different cycle so far. At first I only read the front page because it had enough of content for me, but now I skip many of the front page stories and go directly to the "lost" gems on the new page.

timf• 17 years ago

I hang out in the new section a lot too. I can't tell if many people also do or not, I assumed before that it was a popular place to sift through. But now I am not so sure, e.g. HN founder pg does not seem to mind if the new queue becomes a firehose:

http://news.ycombinator.com/item?id=475779

I am going to miss a good S/N ratio there, though.

yalurker• 17 years ago

One key difference is that HN can be seen as "startup news" not "tech news". This is a much tighter niche, which reduces the appeal to certain demographics. It also means that if HN starts down a "slippery slope" that it starts from a point further away from base, giving more time to adapt/confront it.

run4yourlives7• 17 years ago

Originally, HN was actually called "startup news". The niche was much tighter than what it is now, but it's still relatively tight.

I feel that this is only a matter of time though.

anthonyrubin• 17 years ago

Would discussions of top ten movie lists, politics and excessive meta-discussion qualify as starting down a "slippery slope"?

ilamont• 17 years ago

It's not as niche-focused as many people (including myself, until last week) think it is.

pg has said the focus of the site is "stuff that's interesting to the intellectually curious."

http://news.ycombinator.com/item?id=466881

tokenadult2• 17 years ago

"Perhaps give an actual, real IQ test to use a site. There are several 10- or 20-question tests that have a .85 or higher correlation to full-length tests, and that is probably good enough for internet work."

Suggesting a short-form IQ test as a way to cure the problem of site degradation suggests the author hasn't read many sites populated by people who have scored high on IQ tests. They can be appalling too. Rationality and IQ are orthogonal

http://yalepress.yale.edu/yupbooks/reviews.asp?isbn=97803001...

http://www.cs.berkeley.edu/~russell/papers/aij-cnt.pdf

and there are plenty of people with high IQ scores who don't know how to behave themselves in online discussion. For some really pathetic examples of IQ tests identifying high-scoring ne'er-do-wells, see

http://www.eskimo.com/~miyaguch/

and especially

http://www.eskimo.com/~miyaguch/history.html

for what really happens when high-IQ people get together. (Sometimes those unusual IQ tests are not validated, but then again validation is just another problem with IQ tests. )

To light a candle rather than curse the darkness, I would suggest that the way to keep an online forum on-task, civil, and useful is simply (difficultly) to moderate it well. As I recall, Reddit's "no censorship" policy has been manifested with very minimal moderation. Deal with the user's overt behavior on the site, and don't worry about the user's IQ score. Smart is as smart does.

unalone10• 17 years ago

Wow, fascinating links.

I've never understood people older than, say, 15 who still insist on things like IQ tests. I've never noticed a correlation between IQ and observable intelligence: in many cases, quite the opposite. Many bright people don't apply their intelligence and learn to use it, and then their insistence on their mind belies their ignorance, much to the irritation of everybody else. There's more to intelligence than testing.

lionhearted2• 17 years ago

Raw natural ability can be pretty crippling to development in most fields. But people with raw natural ability who also develop discipline, humility, a willingness to fail/endure/persevere come out to some of the most amazing people of all time.

mixmax• 17 years ago

Mensa forums are actually notoriously trollish, so your point is good.

jm4• 17 years ago

I was about to mention them but you beat me to it. I was subscribed to the GenXMs Yahoo group for a while and always found myself thinking, "A lot of these people are total idiots". The group seemed to be dominated by right wing trolls. I unsubscribed in disgust and rarely attend any Mensa events. I thought the high IQ requirement thing would keep the riff raff out, but I learned by experience that stupidity knows no bounds.

In any case, scoring high on an IQ test really only means that you can score high on a test designed for you. Ever read Isaac Asimov's essay 'What is Intelligence, Anyway'?

http://talentdevelop.com/articles/WIIA.html

latortuga• 17 years ago

Not only that, but where there is an IQ test with questions, there's a Google with answers.

tokenadult2• 17 years ago

The first hacking of the Stanford-Binet L-M IQ test (the one I took as a kid) documented in published reports was by Robert Sternberg, who discovered in childhood in the 1960s that the answers could be looked up in a library. Sternberg has gone on to become quite a well informed critic of IQ tests, although he still attempts to develop easy to administer mental tests to measure intelligence according to his triarchic theory of intelligence. A very notorious example of cheating on that same brand of IQ test came much later when a little boy's mother looked up the answers for the Stanford-Binet L-M before having her son tested at a supposed specialty center on highly gifted children.

http://positivedisintegration.com/justinfu.pdf

Somehow the psychologist who administered the test was unable to tell that the boy was simply providing answers he had previously learned from the test manual. That (and her articles on IQ testing) has shot her credibility with me.

jacquesm2• 17 years ago

"Rationality and IQ are orthogonal"

I'm pretty sure Hans Reiser and the Ted Kazcynski would have aced any such tests.

At the same time there are plenty of people that are not of a high 'IQ' that have plenty to contribute.

So, what you should not measure is some consumer index of intelligence but objective contribution. Tough problem!

bd• 17 years ago

I'm pretty sure Hans Reiser and the Ted Kazcynski would have aced any such tests.

Incidentally, yesterday I managed to stumble upon another such example:

"Nathan Freudenthal Leopold, Jr. and Richard A. Loeb were two wealthy University of Chicago students who murdered 14-year-old Bobby Franks in 1924, and were sentenced to life imprisonment.

Leopold, age 19 at the time of the murder, and Loeb, 18, believed themselves to be Nietzschean supermen who could commit a "perfect crime".

The friends were exceptionally intelligent. Nathan Leopold was an intellectual prodigy who spoke his first words at the age of four months, and tested with an I.Q. of 210.

Leopold had already completed college, graduating Phi beta Kappa and was attending law school at the University of Chicago. He claimed to have studied 15 languages but in reality spoke four. He was an expert ornithologist, while Loeb, with an I.Q. in the 160s, was the youngest graduate in the history of the University of Michigan."

http://en.wikipedia.org/wiki/Leopold_and_Loeb

fiaz• 17 years ago

A more interesting question would be: why shouldn't HN revert to the mean?

EDIT #1: The speed at which I'm being down-voted warrants some further explanation...

The article headline raises a question about the potential decline of HN as a "quality community". The fact of the matter is that with all things being equal, HN is just as susceptible to decline just as much as Digg or reddit - HN follows the same "social news" principle after all.

So then why shouldn't it follow the same principle?

In attempting to answer this question I'm hoping that members will forward ideas to counter this principle in general, given that all things are equal in this space.

EDIT #2 [on a separate note]: I should add that the future of HN (or Digg or reddit) ultimately depends on the demand of the community. If we get an influx of users that want to discuss:

Ruby vs. Python

Emacs vs. Vim

Bootstrapping vs. Funding

Or whatever else comes to mind, then so be it. Btw, we've been good as a community in avoiding religious discussions, so I'm pretty happy to be insulated from the whole Mac OSX vs. Vista debate!

The bottom line is that we can draw parallels to other communities that have risen/fallen based on the whim of the dominant memes that come and go, but eventually the fate of this place relies upon what memes we choose to propagate.

biohacker42• 17 years ago

No matter how hard I think, I can't come up with a reason that would make HN uniquely immune.

Perhaps the only solution is, as pointed out by one of the comments in the article, to continually start new sites.

endtime• 17 years ago

One important difference is that you need 100 karma to start downvoting people. It makes HN more resistant to invasion, though not entirely so. Perhaps people could be required to have been members for a month before upvoting stories or something as well.

Unfortunately, this requirement is less effective if people cheapen karma by not being judicious with their upvotes, as I try to be here much more so than on reddit. Upvotes effectively shape HN; upvoting witty one liners and puns, etc, is the path to becoming reddit.

jacquesm2• 17 years ago

one easy way around that would be to set the various karma thresholds as a function of the number of registered (active) users.

jwesley• 17 years ago

I think HN has a few advantages that give it a fighting chance:

1. It's not a business, or at least a direct one, so the site owners are not focused on growing the audience infinitely. 2. It is not dependent on advertising revenue, so there is no pressure to go "mainstream" to attract bigger advertisers. 3. The members are passionately self policing. Seems like every other day there is a thread about HN going off the deep end.

chime• 17 years ago

I understand Digg has this problem and I stopped being a regular user there over two years ago. However, with reddit, it's pretty much a la carte. You subscribe to the subreddits you find interesting and your home page will only show those. I do not subscribe to the most popular subreddits like politics, pics, business etc. and instead subscribe to smaller, niche subreddits like wikipedia, long-text, and space. My wife subscribes to funny, cute, and health subreddits and reddit has material customized for both of us.

Yes, it is more work to make sure I get to keep seeing interesting things and it does take some time to find more interesting subreddits to follow but it is much better than seeing typical mainstream news on it all day.

tdavis• 17 years ago

There's a very simple fix for this that has been proven to work: charge for admission and moderate incredibly heavily. Charging some small sum, say $10, weeds out lots of younger people (no credit card). Moderating heavily deters people from joining and causing a fuss because they will quickly lose their investment.

Content (and members) need to be pruned constantly. Comments with too low a karma rating should disappear; they're just noise. Submissions that are turning this place into "Economic News" should be removed before they get to the front page and get random up-votes to create even more noise (if I have to flag something on the front page, that's wrong).

I have moderated many forums and though I've never modded a for-pay community, I know strict moderation works. Some people will complain, but the people who come for good content and good camaraderie will thank you for taking out the trash on a daily basis.

bprater• 17 years ago

I'm all for this, too -- but pg doesn't like the Metafilter community, which uses this model.

lionhearted2• 17 years ago

Having a semi-gated community can work. One of the best places for statistics/business/intelligent discussion in baseball is sonsofsamhorn.net - you need to get invited to get a membership, but there's a "Sandbox" that non-members can post to. Anyone can read SOSH, but the quality stays high due to mostly only qualified members posting/discussing.

jorgem• 17 years ago

What if you could just have some settings: 1) I don't want to see any submissions from people who became "members" after me. 2) And, I don't want to see ANY comments from people who signed up AFTER me.

So the site stays about the same for ME. Even though it may look different to someone else.

donniefitz2• 17 years ago

I don't think HN is going down the same path as Digg. It's still keeping it's focus for the most part.

drfloob• 17 years ago

I know a handful of people that have reasonably high IQ's (above 130 or so). Many of them also have infantile senses of humor, short attention spans, and live with their parents well into their late 20's.

It's been said a few times already, but raw processing power doesn't necessarily say anything about a person's maturity, rationality, or interests.

What could work, however, is a Maturity Captcha! When a user submits a story, (s)he's presented with a series of snarky headlines and/or lolcats and asked "Do you think this headline is funny?" or "Isn't the kitty clever?". Or maybe word problems: "Jack wants to buy a Wii, but he only has enough to pay his rent. Should Jack buy the Wii anyway?". An answer of "yes" to any of those would have some negative repercussion (instant user ban, posting ban, lower karma, public humiliation, etc).

Getting past the captcha would be ridiculously easy for 14-year-old geniuses, but at least there will be a constant reminder of exactly what's unwelcome.

On the other hand, aggressive censorship could take care of the chaff pretty effectively. After all, the problem is, generally: "how does a small ruling-class of individuals shape the growth of an entire community". There no sense in deceiving ourselves about what that means: elitism, exclusivism and in some senses, totalitarianism.

But hey, if it's YOUR community, do whatever you want!

queque• 17 years ago

I like the question about the Wii. Here is another maturity filter question which might sound familiar to many folks: "Does it make more sense to you to: (a.) buy a semi-dorky used car and pay off your student loans, or (b.) buy a brand-new car and take on 5 years of car payments and allow your student loans to chase you well into your thirties?"

christofd• 17 years ago

hmmm... community management

1. Nightclubs are an intesting example of the steady rise and decline of new ideas. They attract copycats and their followers and the club dies, because the initial 'intellectually curious' crowd gets run over. Meanwhile, clubs that are overly elite and gated just don't have the style and communication culture. I've seen this, as a music fan, in Munich, Germany, over a period of 10 years. Often places that look like a dump are able to maintain their fans (that's one of the strategies).

2. Gated communities become stale quickly. Silicon Valley culture developed, because it was beneficial to work together with others as the pie was growing for everybody. (That's also why the industrial revolution created a middle class). The academic community is a stark contrast to this, because there is no tenure position out there for everybody. IMHO (5 years work at university) the communication culture is probably in the order of 50% toxic intimidation games (noise).

ChaitanyaSai• 17 years ago

I was wondering out the meta-phenomena of community members complaining about decreasing quality. While it may be valid, doesn't it make more sense to make a more concrete argument about why the quality is going down.

More along those lines here: http://www.discerniblepreferences.com/2009/02/dilution-of-qu...

When sites like Digg and Reddit revert to the mean -- is HN next?

Related Stories