Solving The Hacker News Problem

http://al3x.net/2011/02/22/solving-the-hacker-news-problem.html

by tianyicui • 15 years ago

407 269 15 years ago

tianyicui3• 15 years ago

IMO, a tag system like StackOverflow or Quora seems a good way to go.

tptacek7• 15 years ago

Tags concede defeat, fragmenting one dysfunctional community into a whole spectrum of subcommunities. How will we handle the tag for "good enough for '08 Hacker News"?

dschobel2• 15 years ago

If you can sideline all the people who want to bullshit about how the ipad2 is the future of personal computing or the quality of the writing at TC, you've already won.

tianyicui3• 15 years ago

No, what Reddit does is fragmenting community. How much does the tag system in StackOverflow or Quora fragment them? Tags helped people to somehow filter their already overwhelmed input stream, focus on what they really interest in, and more importantly, ban the items they genuinely don't like to see.

toddh• 15 years ago

That was my thought too, they both have quite stringent moderation, but that doesn't solve the increasing the quality of participation, unless of course it's decline is what drove some people away.

vidar• 15 years ago

Perhaps pg is too busy these days to really tend to HN? God knows I would be if I were carrying his load.

dools5• 15 years ago

Nope, I got an email from him the other day telling me not to editorialise in submission titles after I posted that Nissan Leaf HTML5 extravaganza with the heading "Behold: We don't need flash for anything anymore. Ever".

I'm sure that post itself rankled our OP.

rudiger• 15 years ago

Question: Why'd you give it such a sensational title?

kmfrk2• 15 years ago

Did you forget all about the "More" debacle?

Gods know it was a terrible decision, but it showed some active participation by pg.

dools5• 15 years ago

I must show my HN immaturity here by admitting I don't know what that is :)

tptacek7• 15 years ago

For what it's worth: I feel safe saying that most high-karma users of HN have a variety of severe concerns with it. My experience asking this question over email has generally been one of getting gigantic essay-length responses.

In my official capacity as "representative of people dorky enough to have karma this high", we do officially declare: stuff's broken. Needs unbreaking.

YuriNiyazov2• 15 years ago

Did you forward those essays to the powers that be?

tptacek7• 15 years ago

No. I'm high-drama enough as it is.

JonnieCache7• 15 years ago

If there is one thing that causes an internet community to break down, its when a subsection of that community has too muh communication out-of-band in another medium that the rest of the members are unaware of. Especially if that other medium is more synchronous than the primary medium.

Many forums have been driven to civil war by a clique plotting on a secret IRC channel. (Note: I am not suggesting that is happening here! Just that I've seen it before.)

lionhearted2• 15 years ago

Still my favorite site on the whole internet, but yeah, things have definitely changed.

Stories, yeah, they're a bit worse but that's not the main thing. I skip stories that don't appeal to me, and you can always go a few pages deep if you want more stories.

It's the comment quality sliding down that's scary to me... it's becoming much more okay to engage in standard-internet-level discussion here, which is to say, much less civil than HN used to be.

Perhaps liberally killing any comment that's primarily insulting, sarcastic, pedantic would help?

For instance, I just saw a link to this two year old thread today, look how civil and friendly it is:

http://news.ycombinator.com/item?id=554889

The only halfway insult on there - "Dunno but she has a Twitter page. Not sure why PG would quote such a weirdo." - was voted to zero. And the 2-3 jokes/sarcasm stood still at 1, none of it voted up.

Also, I'd love to see people who just consistently left substance-less comments get put on auto-kill, but I know that'd never happen for a ton of reasons.

I'm thinking of one guy who just posts about how outraged he is about anything remotely politics related, and nothing else. Joined in the last six months. Actually, that probably describes a lot of users now, which might be part of the problem...

DanielBMarkham4• 15 years ago

Case in point: here's a thread from 2 years ago where we all talked about atheism. Civilly.

http://news.ycombinator.com/item?id=236808

You couldn't get anywhere near that today. Discussing tough and interesting subjects civilly with some of the smartest people I know wasn't what brought me to hn (startups were), but it was a big factor in keeping me here. I knew if I posted some crazy half-assed startup idea that might piss people off that the community could talk about it logically and dispassionately. That allowed each of us to open up more.

Those days are gone.

DanielBMarkham4• 15 years ago

In my official capacity as commenting wind-bag, I'll second that.

HN started as a way for YC guys to communicate. It was about startups, the tips and techniques to make one, to grow one. Once PG opened it up, it became "stuff hackers like", which meant cool deep tech stuff. But the tech stuff was there because after a while, all the startup stuff just ran together.

Then "stuff hackers like" kept drifting wider and wider. I was happy with that, but now that the traffic numbers are through the roof, what I'm seeing is the 1% snark factor is out of control. If each member of HN said something truly snarky only once a month, there would still be thousands of snarky comments a day on here. It's just too much noise.

HN has always been driven by emotion: the only reason to click that button is to express your emotional response, good or bad. Sure you can rationalize that in various ways, but in principle it's about how the thing makes you feel.

Now we have the love-fest stories, the hate-fest stories, the snide jokes that get upvoted, the trashing of "rate my startup" posts (I still can't believe how some of the startups are treated). It's becoming less of a community and more of a mob.

I think maybe the story titles might look mostly the same, perhaps, but the quality of the stories and the quality of the conversations have changed quite a bit.

mattjaynes• 15 years ago

Excellent point and thanks for all your thoughtful comments.

I use to comment regularly on HN (member since Apr 2007 and former #1 on leaderboard in the ancient days, haha), but have found that I very rarely do any more.

When I look at my thought process now when thinking about posting something that may be of value, I find myself very reluctant. This is due to the snarkiness and hot-headed emotion that often disregards logical arguments as worthless and instead values breathy spittled anger.

Cowards become brave behind anonymity.

Who enjoys "lawyering" their comments? Basically trying to anticipate and address all the possible concerns hot-head users will bring up. Yet, I feel like I've had to do that more and more. In the end it seems an exercise in futility - particularly when I can just move on to interacting directly with the thoughtful hackers I care about and avoid those that spit when they talk.

I can understand the lament of the OP since I've really enjoyed the community and associating with the gentleman-hackers I've met here.

Thanks especially to those who have added true value here - I've learned a lot from the thoughtful ones who share and help.

NickPollard• 15 years ago

Maybe upvotes/downvotes could be weighted by the karma of the person giving them, thus putting more emphasis on those with high karma (which might sound elitist, but the only reason those people have high karma is because people think they talk sense).

This would concentrate power slightly in the hands of those long time dedicated commentators ('the community'), and slightly lower the strength of newcomers who are still getting used to the ettiquette ('the mob').

Hopefully this would go someway to lowering the effect of newcomers, until they rack up enough karma themselves to 'graduate' into the higher karma ratings.

davidu• 15 years ago

Agreed -- I've always enjoyed my debates with you. I'd go so far as to say I sort of miss them. But I've now grown so frustrated by the pedantic replies that I get on HN nowadays that I've had to start deleting my comments after I write them. And I've stopped visiting as often because it takes too much effort to write a response that I feel avoids the trollish responses I fear I will get.

One of the final straws for me was when the title of my submission about being replaced as CEO was changed -- a topic of significant relevance to this audience I think. What frustrated me most was that I had picked a useful title and the not only was the new title worse in my eyes, but it was surrounded by rather inane posts about the "top X reasons for Y" or something about Apple or Google. It just didn't add up for me.

And on a tactical level, there is unquestionably a tendency for people to respond to the top post rather than start a new top-level thread. The lack of collapsing of threads is quite frustrating on long topics.

For now, I'm mostly done here.

jdp232• 15 years ago

In my official capacity as "a relative newcomer", I agree that stuff's broken. I think one of the challenges the site is facing is a question of how much to continue to cater to the old guard and those who share their perspectives, and how much to evolve? The start world in general (and YC in particular) is at a very different place than when HN started ...

petercooper4• 15 years ago

I also think there are problems (though I've not spoken with you specifically). However, I think this is a universal problem that doesn't necessarily need a solution. Most sites seem to have a group of hardcore participants with axes to grind. Top Wikipedians have lots of issues with Wikipedia. Top eBay sellers have problems with eBay. Even on MetaFilter it's not always plain sailing..

jcsalterego• 15 years ago

It seems there's always been reluctance to add features, for the sake of simplicity. Personal messages, for example, would have been useful in many, many instances, but instead we find ourselves checking out the plain-text profile and finding alternate methods of communication.

This limitation has also sprouted ancillary sites attached to the HN Tree of Life, such as searchyc.com, hackermonthly.com, and hnrecap.com as mentioned in the post.

In a similar vein, carving out a sub-HN seems to be: a) downloading the source code, b) bringing it online at another domain and c) announcing via "Tell HN".

All in all, unless someone with >10^5 karma decides to take the time and add some community features to HN (for various values of "community" and "features"), we're all going to continue and see more noise and many different signals.

As an aside, I wholeheartedly appreciate the name, "Bloomfilter."

jmm57• 15 years ago

As a low-karma, long-time lurker, I'm not sure I've ever really seen the kind of submissions he is looking for. Can someone provide examples of submitted content that would meet his criteria of deeply technical discussion worthy news?

mikeleeorg2• 15 years ago

I keep reloading this comment because I'm in the same boat as jmm57 and would love to see some examples as well.

Out of curiosity, is Hacker Monthly (http://hackermonthly.com/) what al3x and other long-time members want? Or is it the kind of discussion & content seen on a StackOverflow?

paulbaumgart• 15 years ago

http://news.ycombinator.com/item?id=1304354

http://news.ycombinator.com/item?id=1321443

sachinag2• 15 years ago

As a longtime MeFite and a longtime member of this community, I believe that most of the issues could be dealt with by having obvious and active moderation.

As MetaFilter, not only do we know who the mods are, we know which mods are on call at what times. (And there's 24/7 coverage.) HN relies very heavily on a flagging system, but it's just not as responsive to stuff that is broken as is a human who's responsible for what's on the front page and what's in the comments. Having a handful of humans who are responsible for curating the front page (and possibly also pinning really good stories from new onto the front page) would solve most of these problems. Is this less democratic? Sure it is. Would the unfairness be worth it? In my opinion, yes.

This problem just isn't solvable with code; it takes benevolent dictators.

tptacek7• 15 years ago

What's the threshold HN crossed where it started needing active moderation? Because '08 HN threads were just better than '11 threads. Is it just a number of users, past which no set of guidelines restrains pathological conversations? Why? If you can trace it back to what thing happens when you get to your 50,000th user, maybe there's some passive moderation mechanism you can identify to keep it in check.

_delirium4• 15 years ago

I'm not sure it's number of users--- Kuro5hin's downhill slide roughly coincided with a net loss of users, and its heydey was probably also its most populated era.

'08 HN seemed pretty similar to me, though, though I was just a lurker. It was a mixture of good technical content, startup-scene celebrity watching, too many TechCrunch posts, a vaguely political thread every few days (sometimes with a weak "hacking" justification), and the periodic thread once every 3-4 weeks about how HN is dying / has too many political posts / is turning into reddit. Just fewer total comments and votes (by a good margin).

Some random examples: http://news.ycombinator.com/item?id=179755 , http://news.ycombinator.com/item?id=105739 , http://news.ycombinator.com/item?id=109052 , http://news.ycombinator.com/item?id=80234 , http://news.ycombinator.com/item?id=437727

stcredzero4• 15 years ago

Theory on the threshold: the point at which groups that social engineer social news sites decide the site is worth their time. This is also a point where the population has grown to a certain point and the discourse has regressed, providing a larger and more easily manipulated audience.

sachinag2• 15 years ago

For me, I noticed it when people started posting things from investor blogs and the VCs and angels themselves started commenting here. I think that spurred some small handful of users to start showing off how smart they were in hopes that they'd get a response from fredwilson or whoever, and that just naturally spread the way it did. (There's a certain high when you get fredwilson or msuster to respond to your comment on their blogs; I think it's the same dynamic here.) I'm not blaming the VCs at all here - but that's when I started noticing it. (joshu and yegg are the exceptions that prove the rule because they're hackers first.)

Also, I've been told the Something Awful community is surprisingly pretty awesome, although I'm not a participant there. If there's some intersection of structure/moderation between MeFi and SA that HN could implement, I think that'd be a great place to start.

sighs We complain because we care. It doesn't mean that we have all the answers or we're pointing fingers. It means we're so invested in finding a solution to this problem that we won't just move on. Given just how good HN was (and remains), minute changes are more noticeable. I don't think a comparison of then-and-now really gives us useful data. There's just a gestalt, a feeling, you get that something's not quite the same. The closest I can think of is when you're out to dinner with someone you've fallen out of love with; your routine isn't any different, but you just know that it's not the same.

mquander• 15 years ago

I think there's a real disconnect between HN and Metafilter. Metafilter is about making good posts. There are tens of thousands of active members and only a few dozen posts per day because you are supposed to make good posts. The motto is "best of the web" and it is a serious motto.

By design (I think?) HN is not about making good posts; it's exactly the opposite. It's about posting whatever link you happen to stumble across. The hope is that maybe the good posts will rise to the top via voting, and that comments will add value to bad posts. This is a really different idea! I'm not personally convinced that it's possible to get a high volume of visible good posts using this idea in an open general-interest community, because no matter what you do, there will always be more people ready to vote for popularized, lurid, and flamebait posts than good posts.

I think that distinction has much more of an effect on content than moderation. (Although if you've decided that you only want good posts, you probably need moderation to enforce your decision.)

gabaix• 15 years ago

what about tagging? automatic or crowd-powered. Seems a great way to sort through the noise. tags could be "technical", "startup", "YC" etc.

mcav• 15 years ago

HN would be better if it were invite-only like Dribbble. I'm not a member of Dribbble, but it's a good example of why restricting community growth is beneficial.

We're too late for that here. I don't think PG has enough bandwidth or interest to truly solve the problem. New users will continue to join, adding noise to the signal, unless HN changes course. It's going to become more generic and more biased the longer the site stays open.

I hesitate to suggest more moderation as some posters suggest. I'm already uncomfortable with the murmurs of unfair moderation in the system here.

pg2• 15 years ago

Actually HN is the reason I don't have enough bandwidth for other things, not the other way around.

jacques_chester7• 15 years ago

Ah yes, the cycle of website life.

* Hot new community forms at Site X.

* Site X residents refer to themselves as the New Wave of whatever. Much better than older Site W because of features/members/dynamic/demographics 1, 2 and 3!

* Site X's reputation spreads to former hot new sites T, U, V and W. Site X begins to attract more and more new users.

* Site X denizens begin linking articles at T, U, V, W and vice versa.

* Site X begins to exhaust natural topics of conversation. Denizens of more than 3 months standing become sick of 100th "What does Site X think about AlphaGamma?" post and begin to slap down newbies.

* Someone reminisces out loud about the Golden Days of Site X.

* Discussions on Site X become more and more about Site X. Extremely intelligent individuals begin to earnestly argue that their proposed feature will save Site X from itself.

* Someone proposes or launches Site Y. A how new community begins to form there ...

I've been watching this same story play itself out since Slashdot circa 1998.

tptacek7• 15 years ago

This comes across as snide. Slashdot circa 2011 is largely unreadable. So is Digg, so is main Reddit. The cycle may be real, but so is the phenomenon driving it.

jacques_chester7• 15 years ago

Sure. All sites experience regression to the mean[1].

Usually this starts happening about 12 months after I decide to become a regular. In HN's case I've come in halfway through the cycle, about 3 months ago, so apparently the magnitude of my destructive powers is increasing.

[1] and also the snide.

JonnieCache7• 15 years ago

>I've been watching this same story play itself out since Slashdot circa 1998.

And every social subculture in history. See: the constant evolution of the social scenes around music genres. It's just a fact of social psychology we have to live with.

"Dubstep was sooo good back in 02 before the students found out about it! I mainly go to future-garage raves now man..."

mkr-hn10• 15 years ago

And Linux was so cool before the Ubuntu kids showed up.

jacques_chester7• 15 years ago

Good point.

On reflection it resembles flocking behaviour. I imagine that website members can be modelled as simple agents with two rules:

1. A cool site has few, impressive members.

2. Move towards the cool sites.

Given an arbitrary distribution of "impressive" members, this should cause a constantly moving flock of members to move from site to site regardless of its purpose.

tastybites4• 15 years ago

Let's not forget it happens in the actual cities we live in as well! Hipster-ism gives way to gentrification and then the cool kids move because there are way too many yuppies. Same things happens with suburbs - they get too crowded and people start moving to the country to make new suburbs.

This is just human nature, it manifests itself in the clothes we wear, the cars/bikes we drive, the places we live, it certainly also applies to social websites. I'd be surprised if it doesn't - we'll see how facebook fares, or if there will be a coolkid exodus as the rest of the world gloms on.

It's basically a (sub)cultural arms race that everyone fights in.

orblivion• 15 years ago

Big mistake mentioning future-garage, now I looked it up on Google and know what it is. Pretty soon I'll be showing up uninvited to your raves.

mikeleeorg2• 15 years ago

On reading this, the Evaporative Cooling effect comes to mind.

http://blog.bumblebeelabs.com/social-software-sundays-2-the-...

cletus• 15 years ago

While I basically agree with you--I too have seen this cycle many times before--what I don't understand is why people feel the need to slap down newbies in an attempt to recapture some since-passed golden age.

If I get bored of a site or simply get nothing more out of it, I just move on.

If I see topics that are boring or simply have no interest to me, I just ignore them.

If I see rude, insulting comments or personal attacks, I just flag them and move on.

I really don't understand why people need to express so much hate.

jacques_chester7• 15 years ago

Loss aversion, maybe?

jdp232• 15 years ago

People who are deeply invested in a site want to defend their view of how it should be. Change is threatening, and it's easy to react with knee-jerk hostility.

lionhearted2• 15 years ago

> Site X begins to exhaust natural topics of conversation. Denizens of more than 3 months standing become sick of 100th "What does Site X think about AlphaGamma?" post and begin to slap down newbies.

This implies it's hostility to "old hat discussions" that have already been around the block that people are worried about.

I don't think that's primarily it. The civility levels have slipped a lot... this place used to be as civil and friendly as meeting a bunch of people at a dinner party where everyone admires the host.

Still super-civil compared to the rest of the internet, but the levels have come down a lot. Raw outrage, profanity, and outright insults used to get downvoted to oblivion even if people agreed with the general argument the commentor was making. Rudeness really wasn't tolerated, so new members learned quickly that you had to think for a minute longer before replying.

It's still a great site, but it's not elitism and reminiscing that the old guard is scared of. It's comment quality slipping to rest-of-internet level.

derefr2• 15 years ago

How about, instead of a single community, the site runs as a set of "cohort" communities that each only allow direct sign-ups for their first year in existence, but then run a bit like leagues thereafter? If people like you on HN/2011, you might be invited into HN/2010, then HN/2009, etc. It could also be automatic/karma-based: if you earn enough karma on HN/2011, you start again on HN/2010 with 1 karma point (though you can also stay on HN/2011, marked as an alumnus/boddhisattva.

malandrew6• 15 years ago

Pretty interesting idea. Has anything like this ever been tried with other communities?

JonnieCache7• 15 years ago

I think this would just breed resentment, or at worst Lord Of The Flies/Robbers Cave style anarchy.

[1] https://secure.wikimedia.org/wikipedia/en/wiki/Realistic_con...

mkr-hn10• 15 years ago

A recursive treehouse sounds like an idea worth trying.

malandrew6• 15 years ago

Just because this has been the typical website lifecycle in the past, doesn't mean that it has to be the typical website lifecycle in the future. We increasingly have better and better tools to manage how people interact with sites and communities.

Upvoting/downvoting, filtering, disemvoweling, moderation, tagging, reputation systems, etc. are just a smattering of the tools we have available and with time it's likely that we will see more and better tools for community management available in the future.

The most important thing is for "us" to define exactly what current and past features of the community we want to preserve and also decide what kind and amount of "chaos" we want to permit to allow the site to evolve and improve. It makes the most sense for PG, YC founders and very high-karma users to be responsible for defining these community characteristics, ideals and values.

SoftwareMaven2• 15 years ago

The problem is that many of those tools are affected negatively by the regression to the mean as well. An average comment, that would never be upvoted during the early elite, will be upvoted by average people during the mean. Reputation follows.

redthrowaway5• 15 years ago

Interestingly, this isn't a problem related to any particular category of site. Hell, look at 4chan: bitching about newfags and how /b/ isn't good anymore, how /b/ was never good, different factions with different ideas on how to fix the problem (or what the problem is, or if there even is one), spinoff sites where the early adopters migrate to...

This isn't an HN problem, it's an Internet problem. pg could probably do some things to ward off dilution, but it won't last forever. Interestingly enough, Something Awful seemed to be pretty damned successful in this regard by doing the unthinkable and charging people to post there. I'd hate to suggest it as a solution, but it has been effective in the past.

stcredzero4• 15 years ago

pg could probably do some things to ward off dilution, but it won't last forever.

I think a paraphrase of this sentence has appeared in HN comments 100's, perhaps 1000's of times.

astrange2• 15 years ago

SA has internal problems too. There's a constant slow cycle where the individual topic forums become boring and straightlaced because the users get proud of how mature they are, so they start new subforums where people are allowed to make "bad" but interesting posts as long as they pretend to be doing it ironically. Then everyone moves into those and it collapses, because they really did become bad posts.

Sometimes they move offsite instead, usually because the site admins didn't actually like the topic enough to make more forums for them, or because the admins banned all the users because they had formed secret IRC cabals conspiring against them.

The last stage is that the admins change the forum CSS so it's unreadable. I haven't figured out that part yet.

psyklic• 15 years ago

I wonder how different the website lifecycle is from the cycle of nightlife hotspots.

sophacles3• 15 years ago

I agree. One thing tho that never seems to come up: we people from older waves ignore the fact that we spent the last $N years growing, getting better, getting deeper into stuff, while the site has not grown with us. So even if the site hasn't changed, our perception of the topics and such has -- intro to X used to be deep, since we were little, but now that we are big, we just see a puddle :)

A1kmm• 15 years ago

If you are right, a website could automate the cycle as follows:

1) Every user belongs to at most two (or k) classes.

2) Each class can have a finite number of users n.

3) When a class fills up, a new one is opened.

4) There are always two (or k) open classes at a time, and new users join both of those two (or k) classes.

5) The classes are staggered so they don't close at the same time; e.g. if k=2, n=10000, the first 5000 people to join belong to class 1 and class 2, the next 5000 belong to class 2 and class 3, the next 5000 to class 3 and class 4, and so on.

6) Users can only post to classes they are in, and can only comment on posts in classes they are in (or maybe newer ones too).

7) By default, users only see posts to classes they are in (or maybe their classes + older ones). They can opt in to see more if they want.

The idea might work better if it is combined with some more onerous mechanism to join classes late - a fee, a requirement for an endorsement from a member or someone. Charging a fee for access to the more '1337' earlier classes might be a good way to monetise the site; the fees could either be flat, or could escalate (exponentially?) with the age of the class and / or the number of people who pay to join it late (possibly with a discount applied as active members leave).

Classes might get too small due to attrition - coalescing older classes when they get too small might be a solution to this - it might also make sense to increase class sizes (exponentially?) as the site gets more popular.

reedlaw• 15 years ago

Could this phenomenon be in any way attributable to nostalgia? Personally, I find any online community loses attractiveness after a certain period of time.

pg2• 15 years ago

I think if we could see random frontpages from days a few years ago, we'd find that the top stories weren't that different, and that there was the same "jack of all trades, master of none" aspect to the site that Alex complains about. It may be that a site whose design spec is to satisfy hackers' intellectual curiosity would necessarily feel that way.

Maybe I'll write something to regenerate past front pages, so we can check if things are different now. That should be possible, because news.arc has always logged vote times.

dools5• 15 years ago

It may be that a site whose design spec is to satisfy hackers' intellectual curiosity would necessarily feel that way.

As the infamous "How to Become A Hacker" states "The world is full of fascinating problems waiting to be solved"[1]. There is no mention of those problems having to be "deeply technical".

I certainly see the tremendous variety of topics that make the homepage of HN as an asset in keeping with the true Hacker Mentality.

[1] http://www.catb.org/~esr/faqs/hacker-howto.html#believe1

achompas3• 15 years ago

The non-technical questions on HN have poor associated discussions though. As OPs go further from tech/startups (and towards econ and policy), commenters display a poorer understanding of basic concepts.

Bottom line: sure, hackers care about many problems. But that doesn't make up for the poor discussion quality surrounding many non-tech articles here.

bootload2• 15 years ago

A diff of past front pages would be interesting. Some historical front page entries:

2007AUG15 ~ http://www.flickr.com/photos/bootload/1119991071/

2008FEB08 ~ http://www.flickr.com/photos/bootload/2250089864/

2009JAN18 ~ http://www.flickr.com/photos/bootload/3205568708/

2009MAR12 ~ http://www.flickr.com/photos/bootload/3347109733/

2010JUN24 ~ http://www.flickr.com/photos/bootload/4728476663

barrkel2• 15 years ago

Of that sampling, it hit the apex for me with Jan 2009 page. The Mar 2009 page is when there was an influx, and we started posting Erlang stories as a joke to stem the flow.

Semiapies• 15 years ago

Political and other front-page-news posts with little "hacker" content, blogs by irrelevant magazine columnists, posts about how X product/market/phenomenon is dooooomed, random questions about how the site works because the FAQ is a tiny link in the footer...and lots of posts by people horribly concerned that those damn noobs will stink the place up.

I really haven't noticed HN get any worse in the two years I've been around.

gms• 15 years ago

I don't see any difference between the 2007 page and today's.

scott_s• 15 years ago

From August 2007: "Am I the only one who doesn't care about Facebook?"

That has been a theme on HN longer than I realized.

tptacek7• 15 years ago

Maybe you should compare random /active pages from years back. I don't know, I only found out about /active this last month, but I'm guessing the problem will be more visible there.

9oliYQjP• 15 years ago

Most of us that have been around since the beginning (this is my second account) are too busy to participate as regularly as we once did. I'd rather spend time on my business than get hooked on discussions here as much as I did in the past. I will jump in here every so often and I still think HN is relatively great. Doesn't everybody remember the big debate over even linking to Techcrunch articles? We have always been whining about what's wrong to realize that this place has consistently been pretty great.

And sure, we might be like that dance club that's a bit too mainstream. But pretty soon those early cool kids grow up and are also not cool enough to discover the next hot thing. The next HN will come about and it won't be somebody from here who starts it.

weaksauce• 15 years ago

I agree. This place has always been mostly good with a sprinkle of fluff pieces every now and then. The one thing that has changed a bit is the comments. Sadly the comments are being muddled a bit by the new kids on the block. Not that the new kids are doing anything intentionally bad, nor are they really bad comments, but I do remember a time where the people replying to comments were actual domain experts in a certain field or they wrote the application that you use every day, etc.... It's a problem with getting a larger user base. I do think that this site has aged about as gracefully as possible though.

jraines2• 15 years ago

I think this may be due in part to people learning how to target Hacker News readers with their article and blog post headlines, so while the front page would look similar, the newer linked content is more fluffy.

I mean, there has to be some explanation like this, because it just seems so clear from regular reading that the quality has gone down. Then again, maybe I'm projecting the decline of comments onto the content.

mkr-hn10• 15 years ago

The fact that HN hasn't been overrun by marketing zombies is why I'm not too worried about its direction. If linkbait titles are the big complaint about submissions, it's a good sign.

petercooper4• 15 years ago

You (pg) will not need this since you have the data.. ;-).. but for anyone else interested in doing analysis over time, HackerSlide stores JSON representations of the HN front page every hour (starting Oct 15 2010).

http://hackerslide.com/data/YYYY-MM-DD-HH.json gets you what you want. For example: http://hackerslide.com/data/2010-10-15-23.json

metachris• 15 years ago

Wow great! http://hackerslide.com/data/2010-10-15-05.json appears to be the earliest.

Do you provide this as an "api" and allow people to build apps on top of it? If so, are there any limits?

stcredzero4• 15 years ago

I wonder what Google's reading level rating mechanism would make of changes over time? Disturbing thought -- what about comments by poster?

cachemoney• 15 years ago

The frontpage content is fine. I'm much more concerned about the comment quality.

alexknight2• 15 years ago

That's always a serious concern. It's not surprising that many bloggers either heavily moderate or don't even allow comments on blog posts. It's a difficult issue to solve for sure. Better comments means you need better people offering thoughtful and insightful comments.

shii3• 15 years ago

There was a submission a couple of weeks ago of a site that randomly arranged previous HN submissions of a certain threshold from the past with each refresh. Was really useful and sadly neither Searchyc.com or Google is helping with finding the post.

Perhaps the submitter sees this comment and shares the link here again?

sophacles3• 15 years ago

I have recently begun to realize that there are 2 aspects to "website degeneration". One is the obvious new people showing up because they heard it's cool psuedo python paradox situation.

The other comes from the old-timers themselves. I'm not particularly that old-timer, but in the 3something years I've been here, I can say I have experienced this:

I've grown.

The "5 awesome vim tips" or "super deep closure stuff" articles that today annoy me as fluff were deep and new to me back then. HN was a fantastic site introducing me to amazing things, and it is introducing a lot of people to the same amazing things now. As such, the quest for truly new content needs to go deeper, but the people who need the lighter content are greater in number and enthusiasm, ensuring the fluff rises.

When the older crowd doesn't notice their growth, or the lack of change in content (or the combo) you start to get "it's gone down-hill" type comments.

edit: I mean the quest for truly new content for me needs to go deeper, not that the site necessarily needs to go that way.

moultano• 15 years ago

A similar effect: The average age on the internet is younger than you, and will be for the rest of your life. Think about what effect that has on the density of quality user generated content.

wisty• 15 years ago

http://marcel-oehler.marcellosendos.ch/comics/ch/1992/07/199...

Calvin and Hobbes, 16 JULY 1992

" One of the joys of being a kid is that experiences are new and therefore more intense. For example, I'm about to stick my nose in a jar of mustard and inhale deeply! Let's see what it's like. WHOOP!!

See, whed you are oder, you dake your sinuses fo granded.

Some of us prefer to.

jsrfded• 15 years ago

Random highly-ranked hacker news threads: http://www.skrenta.com/hn/

shii3• 15 years ago

Yup, this is the one I mentioned in this tree a little earlier.

dewitt• 15 years ago

There are plenty of cached pages in the Internet Archive:

http://waybackmachine.org/20070221033032*/http://news.ycombi...

The first one saved was on Feb 22, 2007, just after the site launched:

http://replay.waybackmachine.org/20070221033032/http://news....

Notice that the second link is to pg's announcement post.

Second cached page, Feb 26, 2007:

http://replay.waybackmachine.org/20070226001637/http://news....

And so forth:

http://replay.waybackmachine.org/20070405032412/http://news....

http://replay.waybackmachine.org/20070406195336/http://news....

Bounce on the ▸ icon to page from day to day.

This one has a gem on it from paul:

http://replay.waybackmachine.org/20071001102019/http://news....

The archive is not complete, but it does give a reasonable glimpse into the recent past of HN.

[Edit: I think I'll go crazy before getting the line breaks to look correct.]

mkr-hn10• 15 years ago

Sounds like (s)he wants a Less Wrong for startups.

zaidf• 15 years ago

Or may be after a while our perception about something changes disproportionate to the actual change?

That's basically boredom--and it can happen even if you consume something good for a long time. That "good thing" doesn't change so much as your perception of it.

jaekwon• 15 years ago

Hi, I'm taking on this challenge. Al3x, can you send me an email at jkwon.work@gmail.com?

Also, I'm taking suggestions for seed users. There will also be a HN Karma cutoff where everyone above a threshold can join. You can nominate HN users or yourself here.

teyc• 15 years ago

Where is the data that shows HN has degraded? We aren't seeing kitten pictures. A scan of the front page shows the mix of articles being programming, startups, tech.

I'm not sure what Alex wants? More discussion around PG's hackers and painters?

SeanLuke• 15 years ago

This was discussed before: http://hackerne.ws/item?id=1934367

I had a comment which I guess I should link to rather than repost: http://hackerne.ws/item?id=1934605

mickeyben• 15 years ago

Very well written article. I think he has some good points. I'd really like to see a 'deeply technical' alternative to HN and hope he'll find the good guys !

p90x• 15 years ago

This.

malandrew6• 15 years ago

Forking of sites has the same problems as forking open-source projects. It exacerbates conflict, forces people to choose sides, and ultimately both forks typically end up poorer because of members lost.

Instead, the best solution is to evolve Hacker News as a product.

My personal opinion is that we should put Hacker news in the hands of the YCombinator alumni. Founders and first employees (CEO, CTO, lead designer, first engineer hire and first design hire) of YC startups would probably make the best moderators and admins.

In fact, I would say that it's probably time that PG spin off YC as a full-time startup, assigning control of the design and codebase to one talented UI designer, one talented developer and one talented product manager.

For the site to keep growing in a way that maintains quality, it needs more functionality that it has. The two features that lack the most are filtering and combinatorial game mechanics.

Filtering is necessary so it is easy for the the hardcore tech articles to be easily found by high-karma members, so they can vote those articles up. If it's not findable, it's not voteable. Filtering is also necessary for people to extract the most value out of hacker news. Most users don't want 100 front-page articles everyday. They probably want 10-20 of the highest value articles. Less is more.

Combinatorial game mechanics like those on StackOverflow would help as well. Upvoting/downvoting is limited in that it will always fall victim to the masses. Giving special voting/tagging/burying rights to distinguished members (very high-karma users and YC founders and employees) would go a long way to helping eliminate the crap.

I think I speak for most members here, when I say that I don't want Hacker News to be a democracy. I want it to be a technocracy. I want the smart and accomplished people to control what is good and should be visible to all. I've got only 260 karma points, and personally I don't think that should be enough karma points to allow me to upvote a submission. 500+ karma points should be the threshold to be able to vote an article to the frontpage.

Dylanlacey• 15 years ago

Even game mechanics have a problem.

Say your valued members have a certain mindset, ethos and style. Let's call it YC-meme. Everyone with YC-meme contributes in a manner that exemplifies YC-meme. If someone comes along and doesn't have that meme, their posts tend to not be what the memesters want. So, members use their voting powers to reduce that person's karma. Eventually they either leave, get ignored by all, or become infected with YC-meme.

Now, open up the community to everyone. At first, YC-meme is still dominant. Only 1% of people without it are able to write posts in a style similar enough to gather karma. Let's say they have RD-meme, which has about 80% overlap with YC-meme. Great.

The problem is, soon, there isn't 1% of the community with RD-meme, there's 10%. Who can (and do) re-enforce RD-meme posts, because it's what they want. People can even get lucky and simply contribute 'good' posts, long enough that they get a decent karma score, and so can re-enforce different memes.

What makes it WORSE is that what's acceptible is what's visible, so with more RD-meme posts, RD-meme posts become more likely, as the probability that they get 'punished' decreases.

End result: Community-dilution runaway.

There are a few 'cures' I can think of, but most are fairly prescriptive:

1. A concrete set of rules and behaviors that anyone can reference to moderate from. Gross. 2. A very high bar for moderation, with active participation. Given that the people most likely to have YC-meme are also likely to be busy, this might be doomed by design. 3. "Proof" required for contribution. Great way to kill off your community. 4. Invitations. Either so easy to get as to be meaningless, or so hard as to stagnate the community. 5. Close ranks to just people at/gone from YC. Sadly, that'd mean I was out. May still be the best alternative.

Maybe the community is self-aware enough to start consciously moderating better. Perhaps a post like this every so often might serve as a gentle reminder. That'd be the best outcome, but needs to be chosen by a majority of the user base. Do YOU want a stricter HN?

silvestrov• 15 years ago

2. A very high bar for moderation, with active participation.

The reason this is doomed is: there is currently much more stuff needing moderation than the selected few can handle. They would soon wear out and you'll end up with a site with almost no moderation at all.

The trash needs to be stopped at the front door. When it gets into the house, it takes too much effort to clean up.

erikpukinskis2• 15 years ago

both forks typically end up poorer because of members lost.

I strongly disagree. Off the top of my head: XFree86/Xorg, Mozilla/Firefox, Debian/Ubuntu, Rails/Merb... these were all cases where the existing community had calcified in some key area and a fork was necessary to create a space where important work could get done.

And in most of those cases, not only was the fork stronger than the trunk in (some) specific areas, the fork actually led to the trunk getting stronger. Not only were patches merged, but entire projects, community values, development practices, etc were.

XFree86 is an exception, but organizational paralysis was what necessitated the fork, so it's not surprising they were also unable to respond to the fork.

But I just don't buy your notion that forking is somehow unhealthy and destructive. It's one of the most valuable ways we have to keep open source code, ideas, organizations and practices fresh, healthy, and vibrant.

malandrew6• 15 years ago

I agree that there are exceptions. In the case of HN, I don't see how forking would be of benefit, especially since HN is so strongly linked to YCombinator, the startup incubator/school that tends to attract the most talented up-and-coming startups.

doron2• 15 years ago

The Illustrated Guide to Flame Warriors is a handy reference: http://redwing.hutman.net/~mreed/index.htm

jpwagner• 15 years ago

Well, obviously posts/discussions like this can actually be contributing to the "problem" as some see it, but I'll make one point I don't see made here.

For me personally, I've learned a lot and grown a lot over the course of the 4 years I've been lurking and occasionally contributing here. So for me, a smaller percentage of the stories/articles/posts/discussions appear as insightful as they once did. I don't mean to knock HN in any way, in fact my point is that that fact is not a "problem" to me. New users are joining everyday and everybody who makes the effort to learn and contribute gets something out of HN.

It's what brings me back 17 times a day.

redthrowaway5• 15 years ago

All of this has happened before, and all of this will happen again. HN is succumbing to the problem pg tried to address: dilution. Thing is, reddit already came out with an excellent solution with their subreddit system. This simply wouldn't be an issue if there were different sections for tech, hacking, programming, startups, science, finance, and general interest. Keep all of the deeply technical stuff in one place, the cruft in another. Let's face it, the people who are complaining about lack fo deep tech are also likely to read and enjoy one of Spolsky's blog posts. There's no need to ban the latter to protect the former, just keep them in separate sections.

Now, HN isn't trying to grow, so there's no need to have user-created subreddits (sections, I suppose). Just make 8 or so that people care about, and add another if there's sufficient demand.

I really shouldn't be crediting reddit with this, as the solution existed long before them. All HN needs to do is follow the forum model and have different sections. It's too big to only have the front page.

lwat• 15 years ago

We have become exceedingly efficient at ruining online communities.

dschobel2• 15 years ago

How to solve the signal/noise problem? Amplify the signal.

Call it undemocratic, but insight and perspicacity is not uniformly distributed so it's absurd that pg/$whoever_you_respect's upvote on an article counts as much as anyone else.

As a simple experiment, it would be interesting to see a view of the frontpage based only on the upvotes of people who are above a certain avg-comment karma threshold (since the site is predicated on karma as a quality indicator) and the idea that people who write insightful comments won't upvote crap stories.

groby_b7• 15 years ago

Hm... Story upvotes modulated by users Karma might be good enough. It's an interesting concept...

Stackoverflow operates under this principle, more or less, but it doesn't seem to entirely stop the slow decline.

SoftwareMaven2• 15 years ago

The problem with using karma is that it is affected by the regression to the mean. As more "average" people come in, more "average" people will give them karma. Solve that problem, and you might have a solution to the whole problem (I, for one, am not smart enough for that ;).

mkr-hn10• 15 years ago

Maybe some mix of karma, account age, and the number of upvotes received from first-year members to give a person's votes weight.

richchan• 15 years ago

Alternatively, perhaps we can let users select (follow) a set of like-minded people and make the up-votes of the people they pick weigh more on a "customized" front page.

But then people may have not like having their up-votes be so transparent..

jackolas• 15 years ago

But the interesting issue here is that those who write interesting commentary may not be good at discerning good quality. (according to whatever standard we're holding)

tianyicui3• 15 years ago

This problem applies to all communities using votes to sort user-generated contents: HN, Reddit, StackOverflow, Quora, etc.

I think something like PageRank can really help. The same need is to rank things based on a directed acyclic graph. The only difference is we have multiple kinds of vertexes: users, submissions, comments.

suraj• 15 years ago

What about using baysian filtering? We already have a lot of data on good and bad comment styles, baysian filtering could give us an indication if a comment is violating HN guidelines.

I am not so optimistic about submissions because the data we currently have is tainted by submissions we don't want. However by using the technique you have described, we could probably achieve better results.

arghnoname2• 15 years ago

If you really wanted to make the site more like it was, you could look at the voting history of people here at some distant time in the past and find other members whose votes highly correlate with the old timers. Some people just quietly upvote, so they don't have much karma. If you want the upvoting to be more like it used to be, award people who vote similarly like how older users used to vote.

It would help take care of the effect where older uses leave due to changing interests etc.

Note: I don't actually think it's a good idea. I think just figuring out _what_ you want exactly, in vaguely quantifiable role, is sort of a hard problem. I suspect a lot of people have different 'golden ages' in mind when they think of HN.

When I came here originally it was pretty startup heavy, which was sort of remote from what I was up to at the time. I liked it when it got a little away from startup business related stuff (my perception) and toward more generic technical issues. My point is even if everyone agreed that ait has gone done hill (an open question), not everyone might agree on what it should be like. It's not a democracy of course, but even as an individual for my own preferences that seems as hard as the technology issues.

rubashov• 15 years ago

The basic complaint is that social sites grow into mobs. The solution is rather obviously: halt growth. You have to limit the number of active users to some vaguely Dunbar-ish number or you inevitably wind up with a lowest common denominator mob.

Metafilter did this, right? For a couple years they said "No new accounts."

I think scaling a social site to a very large number of members without deteriorating badly is impossible. It's a matter of human nature and mobs.

petercooper4• 15 years ago

It's definitely not what Alex is semi-proposing but I've been running RubyFlow - http://rubyflow.com/ - for a few years now and it totally stole the MetaFilter model, just for Ruby-only stuff. No "votes" and points scoring - just interesting posts from people in the Ruby community coupled with me editing posts for format and deleting anything that's blatantly spam or offtopic. Seems to work though I have been tempted to go in the voting/Reddit/HN direction with it.. maybe I shouldn't!

mechanical_fish2• 15 years ago

So long as you have the personal bandwidth, you will almost certainly be a better editor than any algorithm you can come up with.

Thanks for the link, by the way.

Mz4• 15 years ago

I'm probably one of those darned newbies who isn't a real hacker and is screwing the place up. (Sorry.) So I wasn't around in 2008 (or whenever the Glory Days were). I don't feel like this article or other discussions about the issue have really given me a good idea of what HN supposedly once was that it isn't anymore. I wish I could get such info. I think that kind of information would hold out some hope of figuring out a real solution -- a means to raise the bar or deepen the discussion or whatever it is that people are wanting.

I know there are other large forums on the internet but this is the largest one I have personally participated in. I think such large forums are breaking new ground, socially, in ways that do not compare to sites like Facebook. Where else can I actually speak with my 80K closest friends? If I am in a room of 500 at work (and not on the stage, because I am not one of the big wigs), only a handful of people around me can hear anything I say. We all can listen to the presentation, but we cannot converse. Here, any and all of us can converse. It is unlike anything you can do "IRL". I suspect that is part of the issue: No one really has a model for how you manage that kind of social interaction. And the models we do have break in that setting.

Just thinking out loud.

groby_b7• 15 years ago

Part of "the glory days" was a much more interesting debate. Currently, lots of HN comments are either going for the cute/snide/sarcastic joke, or argue dogmatic points.

Worse - the snide remarks get modded up. Being guilty of them myself, let me post an example: http://news.ycombinator.com/item?id=2202958

The discussion was about Google's two factor auth. Somebody asks for a good API, poster #2 suggests OpenID. I joke "you read the part about 'good' API". It's by far the highest-modded comment I have. (Which makes me feel guilty every time I see it). Sure, I don't have a lot of comments or karma, so it might be a fluke, but it exemplifies what's wrong with the discussion.

HN'08 would've modded me to 0 for that and moved on. HN'11 rewards playing to the audience.

Mz4• 15 years ago

A couple of real quick and dirty thoughts:

A) Meatier comments/debates require a certain level of trust that the debate is about ideas and not about pecking order. That is enormously hard to achieve in most settings. It gets harder when the community grows. A smaller group is much more able to know each other and all that.

B) My best understanding is that formality is what older cultures with larger populations moved towards as a solution for such problems. I know I find some stuff at work endlessly annoying because of the disconnect between the setting and the social behaviors/assumptions. My mom is European and my dad was career military and my ex husband was career military. Europe and the American military are both more formal cultures than the American Deep South where I grew up (and live currently). I find some of the assumptions of more 'casual' cultures to be very uncouth.

I don't know how to raise the bar on formality for a place like HN but my best guess is that would help with this particular issue. I like a good, healthy debate. I rarely engage in it because most "debates" are really fights (ie they are "arguments" in the other sense of the word -- emotional, social, ugly), not intellectual discussions.

Anyway thanks for tossing me a clue. That does help me think about the issue, unlike most of what I read in these discussions.

yummyfajitas• 15 years ago

The snide remarks got voted up in 2008 also. For a long time, my top rated comment (posted in 2008, 50 upvotes) was nothing but snark.

http://news.ycombinator.com/item?id=318690

(In contrast, my most valuable submission to HN ever only got 14 upvotes: http://news.ycombinator.com/item?id=250027 )

danenania• 15 years ago

I'm fairly new, but I think HN is great. The level of discussion is way, way higher than other similar communities I've seen, and I think the general focus on tech issues with splashes of other topics is perfect. Seems like a successful self-regulating community if I've ever seen one. People just like to complain.

johnrob• 15 years ago

Another reason HN may be boring: we've beaten a lot of the common topics to death. It only takes a handful of articles about "how to pitch a VC" to soak up most of the relevant advice on the subject. While posts often present a unique combination of previously mentioned ideas, it's becoming increasingly rare to actually find something new if you are a regular here.

robryan• 15 years ago

This may also explain why even older members are posting things further away from the core hacker news topics. Something about health care or law or economics might seem more interesting than a pitch a VC article because we have all read many many opinions on the latter.

I think good technical stuff though is evergreen because there are always new languages, libraries and techniques.

erikpukinskis2• 15 years ago

It's worth noting that this is also true in Magazines, where they have strong editorial review, and an actual staff of people creating content.

Once you've read 12 months of Runners World, you've basically read every Runners World that has been, or ever will be, written.

Pooter• 15 years ago

The solution, ultimately, is for the site to wither and die, and be replaced by something else that will have the same fate. This is what happens to all things, and to all human social groupings in particular, from ancient Rome on down to your nuclear family.

If you're tired of it, start something else. Or hang out and jump ship when the next great thing comes along. Trying to preserve the golden age is rearranging the deck chairs on a sinking ship.

jefe78• 15 years ago

I've come to realize in my short time here, that dissenting opinions are dangerous. I've learned to respect the karma gods and pander or, post my opinion and delete it before taking too hard a karma hit.

Its sad to see that an informed, but non-conforming opinion is taken as fact and karma-nuked.

tomrod2• 15 years ago

Agreed completely.

astrange2• 15 years ago

Please cite examples. Your post is practically an example of karma-whoring without them - anyone can make this claim.

peterbraden• 15 years ago

I think that if you stay at any online community long enough, you begin to perceive a drop in quality - even if that drop does not exist.

IMHO opinion, there is plenty of signal in the stream. What has happened is that the interests of the community have diverged. I'd be far more interested in ways to focus on things that I was interested in, within the stream, than narrowing the flow of information.

On my wishlist is a way to pipe the HN stream through a Bayesian filter based on articles I've enjoyed, and make an RSS feed of articles I'd be interested in.

doron2• 15 years ago

Gated communities are effective means of preserving the identity of communities, they all employ some bar of entry whether racial, religious, ageist, or economical.There are many social maladies that are also unique to the gated community, the insularity often breeds all sorts of creepiness. Preservation all to often morphs to Stagnation.

Artists are often the shock troops of a neighborhood gentrification, after the studio loft, comes the artisan coffee, some renegade youths, a young lawyer or two, and before you know it, the neighborhood just ain't what it used to be.

I would Posit that a website calling itself "Hacker News" immediately opened itself to all kinds of interpretations. The term "Hacker" seems to be as hotly debated as "Artist" and justifiably so.

The Hackers, introduced others who identify with the Label, and still others who probably do not, but nevertheless find it of value to their venture.

When the neighborhood changes, you are free, within your means, to move to another place. Sometimes you yourself change and require a change of scenery.

When a startup grows to a full company, many times you lose something while gaining another, and vice versa. Many in this forum have made those choices on their own, so it should be familiar ground.

It is almost heretical to mention it here, but perhaps there is no algorithmic solution (if there is a problem) to the complexity of human relation, expression, and motivation.

More people, more heat, Entropy.

mthreat• 15 years ago

Case in point - this article.

bootload2• 15 years ago

"... I think HN does a crappy job with general tech news and a so-so job with content that’s specifically relevant to startup founders and employees. These days, HN does a downright terrible job with deeply technical topics; that’s the area I hear the most complaining about on Twitter and in private. Since deep tech is HN’s weakest point, let’s go after it. ..."

The weakness of the argument is that the engineer/developer/programmer view is a subset of the interests hackers, founders and entrepreneurs. I draw a clear distinction between tech guns for hire who only want depth as opposed to those who want to solve technical problems and maybe innovate which requires both depth & breadth.

d0m• 15 years ago

Maybe experienced HN (read as a mix of high karma + there since the beginning), could have a bigger impact on which articles are chosen. I honestly don't mind a "dictatorship" selection where chosen members could remove useless post / select useful one.

hammock• 15 years ago

Everyone here has seen this same lifecycle play out at just about every online community there ever was. Doesn't matter whether it was open or closed. It's a fact of life.

The solutions offered are top-down culture modification and just plain don't work. Adapt, and wait for the next HN to come along. You can't stop the train.

tomrod2• 15 years ago

Wait... he complains on his post about not submitting his blog posts to the community.. which then gets submitted to the community and upvoted (albeit probably not by him). Does that sum it up?

kedi_xed• 15 years ago

It's simple really. Digg was good, then it got popular, Reddit was good, then it got popular. I've increasingly visited HN more than Reddit to get my old Reddit fix, as I assume others have, and so popularity has increased and now the quality is degrading as people want their karma fix or 2 cents.

There should be a brainstorm on this. I'm starting to realise I want comment submissions from well known or quality submitters. Not just your average kid or someone who is trying to troll.

The other issue is one-off opinion pieces on some guys blog. HN feels like every programmers chance at 15 mins of fame. Why Ruby On Rails is X times better than this (adudecodingblog.com), My way of speeding up Python (pythonlover.com), etc. having someone like pg, of Joel, or big wigs viewing items or articles like these, offering actual real world advice, and providing comments.

Maybe a subscription based hackernews, where the kudos goes to the legends of the industry, interns are made, and I get my intelli-fix and boredom disguiser because I'm stuck in a cube-farm polishing PL/SQL wondering how the hell I got here and when can I play that stupid COD:Black Ops with its really crappy hit detection. Why do I keep playing it?! Why haven't I asked for a bigger paycheck? Why am I not contracting? How is it that the kid I use to teach programmer is now earning more than me? Oh well, keep surfing...

dave1619• 15 years ago

Another HN challenge... as discussions get longer (like this page), it gets more chaotic and more difficult to follow.

icey• 15 years ago

I would very much like to have http://startups.ycombinator.com/startups/ and have a more narrow focus on startup related news and entrepreneurship.

The constant stream of front-paged political arguments and noticeable increase in mean-spirited commentary in the threads has caused me to spend most of my time on HN logged out. It used to be that I'd read the comments before I'd even read the story to see if the story was worth reading. I wonder if it's possible that pg doesn't notice the degradation in comment quality as much because the trolls have been here baiting him since the very early days.

I don't think HN is irrevocably broken; I'm glad that pg is helming the ship and I think he's doing an admirable job of it so far (I think the ranking algorithm in use for the front-page stories is one of the best anywhere). But HN used to be great, and now it's merely good.

I think that a lot of people who have been here for a long time have thought about what's changed here, and how it could be fixed. I know I've littered more than a few mailboxes with lengthy emails about what I think is wrong, and what I think the solution is. Reading this thread kind of tells the story - a point has come where the community is large enough to have factions that value different things. "Anything that good hackers might find interesting" works when you have a small group of people engaged in conversation. It's less useful when you have mobs of people who have come with different ideas of what they want to get out of this site.

In the early days, HN felt like it was a problem solving tool; a way to find out what cool things people were working on, and occasionally to ask for advice. The community was humble, competent, and full of people who actually made things. Those people are still here, but there's a self-aggrandizing element here as well. The group of people who seem to think that someone else's success somehow reflects poorly on themselves, the bloviators and blowhards who believe that a volume of arguments somehow makes up for the measurable factuality of arguments. I don't really know what the solution is to this. I thought if there was a way to ignore people it might make a difference, but after some experimentation I think that that's a dead end - there is too much chance of missing something truly interesting from doing that.

All this being said, HN has had an immeasurable positive impact on my life - The people that I've met through HN (both in person and virtually) are some of the smartest, most amazing people I've known. I'll get to use the things I've learned from HN (and more importantly from the people in it) for the rest of my life. I can't think of another site on the net that has come even close to making such a huge impact.

I can't imagine missing out on all of this if HN had been invite-only when it launched. I didn't know anyone when I first came here. I didn't even know who Paul Graham was.

Instead of complaining about it, I think those of us that have been here for awhile owe it to pg to actively try to improve the community. It's become too large for him to handle on his own. Yes, there are moderators, but they're an invisible hand that only act as a corrective force.

We're a creative lot. I'm sure we can figure out some way to improve this community from the inside.

pdaviesa• 15 years ago

Pretty soon you guys will be telling the kids to turn down that damn music and stay off your lawn :)

alexknight2• 15 years ago

Honestly I'd love to pay for a Hackernews account if it meant weeding out some rather distasteful people. Not saying that is the be all/end all solution to the problem though.

wyuenho• 15 years ago

Why can't HN force categories on every posts? Have the community create and curate the categories, and select their own categories on HN to read. Just a blanket vote up/down button hardly measures how valuable anything is for any particular group if that group is not constant.

georgieporgie• 15 years ago

If sites want to claim some sort of community and continuity, they're going to have to place newbies into virtual reeducation camps. Want to see the newest links? You have to read through 10 comments from an '07 post first. Posted a link to an internet meme? Back to the virtual reeducation camp with you.

doorhammer• 15 years ago

If the crowd has cycled so much, I wonder if maybe this isn't the best solution for the desired outcome.

Granted, I haven't been visiting tech-specific boards for more than a few years, but I'd generally agree that the more technical articles are what I'm interested in.

I think I'd be interested in a board that was geared toward programmers/hackers, but didn't use a typical karma/point system. I'd like to see one that perhaps utilized karma, but under a collaborative filtering system. So, in a simple for-instance, if a small subgroup of people tend to upvote articles that I do, those articles would be given more weight, and similarly those who downvote articles I upvote would be, from my perspective, given less downvote weight, while at the same time there might be a different subgroup that was weighted to value their downvote more. Perhaps give people the ability to tweak the tolerances of their collaboration. Give them the ability to say "if this guy has X karma and ignores someone's articles and votes, then I want to ignore them too"

Of course, this might be 1. a completely naive idea, 2. an idea that's already been tried and failed 3. an idea that's already being used 4. something to time-consuming for people with real work to do or 5. an idea that's unworkable and that I'm only having because I just started reading books on, and experimenting with, machine-learning ;)

Though even if it existed, I probably wouldn't use it. I already waste half my day reading the few articles that interest me on hacker-news, heh

it sucks that when you design any system or any set of rules, and humans are going to interact with it, you have to think "how are these shady bastards going to subvert my beautiful creation?"

krschultz• 15 years ago

In theory, centrally planned things make a lot of sense.

In practice, democracy usually comes to a better solution, even if it is not perfect.

HN is driven by votes, the community is getting what the majority wants right now. The only way to really improve HN is to change or limit the community. You can tweak the rules only to limit certain actions to high-kharma users, but if there is pent up demand for some kind of story it will make its way to the front page.

JoshCole• 15 years ago

One thing worth noting is that reversion to the mean doesn't have to be a bad thing. For example todays mean level of education compared to the mean level of education a few hundred years ago is very different. A good question to ask might be, would submitting this increase the mean level of discourse on Hacker News? It is the same sort of thing as what is in the guidelines, but reworded for greater relevance.

mkramlich2• 15 years ago

My suggestions for tweaks to improve the site:

1. hard ban on purely political news ("Egyptian leader stepped down! OMG!")

2. hard ban on gender-specific things ("i'm female, went to bar during hacker conference, got groped, OMG!" -- yes it was hacker conference, but gosh subtract the 'during hacker conference' and you have real life, it's independent of tech, not specific to it or due to it, just a life thing with guys and gals)

3. particularly if hard bans (enforced by a set of trusted admins) on the above topics are not added, then allow submitters and admins to add/edit content tags for each post; then allow logged-in users to submit content filters so that when they see, eg., the front page, it can suppress all posts with certain tags (eg., pure-politics, gender, sports, religion, etc.)

4. optional for-small-periodic-fee premium accounts, which allow those users to exercise extra features like smarter content tagging/filtering, sorting, user following, user submission/comment filtering (so you can blacklist blowhards and pedants from what you see, even if they are not banned from the site overall)... I'd personally love to blacklist anybody that ever does a comment reply to me that is (a) rude, or (b) idiotic, or (c) overly pedantic (some is fine, we're nerds, goes with territory, and some precision is valuable, sometimes). Blacklists could be flat files, one user per line. We could share them among each other privately. I've bookmarked a few "ahole-or-idiot" users but I'd love it if I could have them automatically stripped from anything I see on HN in the future. Actually, I'd love to have this feature on all social/forum/news sites I visit.

5. fix the "type comment, hit submit, get error page saying something doesn't exist, so you have to go back, copy your text, hit Refresh, paste the text back in, hit Submit again" bug/feature. that drives me nuts. feels like impl side-effect rather than intentional UX

6. don't have the up/down arrows so close together when viewed on iPhone

7. don't allow just anyone to downvote any comment. or at least, they can't downvote it beyond 1 point, below which is penalty land. right now, any dumbass can downvote a comment of mine from 1 to 0, which then reduces my overall lifelong site karma by 1. Just because they disagreed with me. Or they're an asshole. Or they accidentally hit the downvote button (see 6). Instead, have a minimum karma requirement to issue downvotes, and/or only admins.

HN is great, despite it's imperfections. But I'd gladly pay up for premium features. HN Gold? HNGold.com (YC-W11)?

EDIT: added a few items

spicyj• 15 years ago

(In response to 7,) you already need 500 karma to downvote anything.

hasenj• 15 years ago

I also find I can't downvote direct replies to my posts.

maxklein• 15 years ago

My suggestion would be simpler : have a "this is not hacker news" queue. It's like the frontpage, but it's for the stuff that is "not hackernews". Whenever something hits the frontpage that people feel does not belong, instead of whining, they just hit the "this is not hackernews" vote button. With some algorithm that can combine vote numbers, influencers or whatever, when it crosses the threshhold, it gets moved over to the alternative page. So there is the community moderated hacker news and there is the alternative everyone hacker news.

mkramlich2• 15 years ago

good idea. i've been using 'flag' for that now, but your idea is better because it more precisely signals why one is flagging something. makes distinction from reasons like obscene, duplicate, etc.

nhangen• 15 years ago

I just want to speak up as a relative newcomer that feels I've learned enough about this place to speak my mind without being afraid of retribution and can do so with a basic understanding of what works/doesn't work here.

I really like it here, and it's my 1st stop after Gmail every day, and often more than once per day. Nothing is perfect, but as far as I'm concerned, this is as good as it gets.

ddkrone• 15 years ago

Reeks of elitism.

T_S_• 15 years ago

Put some teeth into karma. Make more karma mean bigger upvotes and downvotes, say 1 extra point for every 500 points of karma. It doesn't have to be linked to when a person joined. It's elitist like a journal or university, but at least anyone can read HN, and good posters will rise. Problem solved?

EDIT: Oh yeah. 1 week comment lockout for negative karma, with a grace period for newbies to learn how to comment.

edw519• 15 years ago

                 Quality of HN Comments Over Time
   |                   . .
   |                  .   . 
  q| . .             .     .
  u|    .           .       .               . . .
  a|     .         .          .           .       .
  l|      .       .              .      .           .
  i|       .     .                  . .               .    
  t|        . . .                       you are here -->. .
  y|                                      (that's all)
   |________________________________________________________
    N D J F M A M J J A S O N D J F M A M J J A S O N D J F
       '09                     '10                     '11

(It must be that time of year again...)

http://news.ycombinator.com/item?id=926604

http://news.ycombinator.com/item?id=1646871

oconnor0• 15 years ago

The question is, of course, are we entering the Eternal February?

buro9• 15 years ago

As of now (9:10GMT) I've read every post on this page, and I think as short as it is, the parent (Eternal February) perhaps is the one that is closest to what I think has happened.

I think this is a people problem, not a technology one.

For those who haven't heard this phrase before, the "Eternal February" apes the "Eternal September". Pre-1993, each September would herald the start of a new university year and an influx of new students who would suddenly gain access to the internet. Whilst their presence was being integrated by existing on-line communities, the standard and quality of debate in Usenet was perceived to decline as the same questions got asked again and the conversation standard was forced again. The same problems came forth until such a point that the existing communities had assimilated the newbies and by October or November things had returned to where they were, just a bit bigger.

Then along came AOL, giving out free/cheap internet access to all and lowering the technical barrier to getting on-line. A good thing for the world, but a bad thing for those Usenet communities who suddenly had to assimilate an influx greater than a normal September, and also an influx that didn't subside. At some point the ability of those in Usenet to assimilate was surpassed by the people getting on-line and finding Usenet. The term "Eternal September" was coined to describe this problem and the general failure of Usenet to keep up with the flow and the lowering standards that are the result of this.

I think HackerNews has no technical problem.

The discussion around karma and mod points is a distraction, the same goes for invite only systems and "classic" view.

I'm going to speculate that the problem is purely the size of the population and the growth rate, and that the community gravity is now such that this is not going to stop.

Anyone who has been here for years, and anyone who has a shed load of karma points... those who give in time, effort... there's no questioning that we/they all want to have a place such as this, and few really want to leave it. But it's hard to see how staying is going to help, and here's why: Time is valuable to us, and our expectations have risen as our experience and knowledge increased, and we feel defeated by that influx and we sense the gravity (that if anything increases the rate of influx).

Due to the growth, the noise level is so much higher (even if only 0.1% of the population post new things or respond, 0.1% of hundreds of thousands rather than thousands is still a far higher noise level). How can we cut through the noise to find the good stuff (subjective to the individual) given this noise? I read the firehose (/new) in an attempt to not miss the good, but realise that I've amplified the volume of everything to try and not miss the gems.

The influx is such that it feels non-correctable to some degree. We can look back at the side-splitting Erlang day as a humorous attempt at a correction (which it was, but we probably didn't perceive the bigger picture then... the "Eternal February").

The thing is, most old-timers are aware there is still a lot of value in learning from new people, of passing on knowledge to new people... it's a two-way street and some good comes from growth and closing the gate isn't the desired thing. As the noise levels get higher closing the gate seems tempting, but it's silly. Shutting the gate is not going to help.

There is a natural contribution rate at which the noise levels are manageable, and there's a natural growth rate at which people can be assimilated such that the noise doesn't go above manageable levels. When either rate is crossed it can be difficult, when both are crossed we end up here.

My suggestion would be along Alex's lines... to not divide by date registered (just those with early registration dates), to not exclude by network (an invite only boys club), nor to kill HN (which still serves as a really important hub by bringing us all together, and I'm sure some would feel that anything new is a conscious threat to the old).

Instead we should create some thing (or multiple things) that is so much more narrowly focused that it doesn't have much appeal beyond a smallish number. That focus needs to be narrow enough that growth is naturally limited too.

I think that defining that 'thing' is where it gets interesting. What that narrowly focused topic is, well that is going to be different from person to person.

It's likely that many small communities would spring up; maybe one focused on devops + computer science (a bit like the ACM SIGACTS-SIGOPS), maybe one focused on purely the business side of startups, etc.

HN is rare because it encompasses all of these, but I'd suggest it may be better to not be so far encompassing and instead to have a set of communities (subreddits vs stack overflows vs unconnected sites) more narrowly focused and people would join subsets that interested them rather than have them all exist in a global scope (where the noise level is that of the whole set).

The hardest part of doing anything comes down to the people.

Ultimately it's the mix of people here that is so awesome, it's what acts most as gravity to those outside of the community. Having this calibre of people choose to invest in smaller communities is going to be the hardest thing when staying on just HN, or just opting out altogether, is so easy to do.

I too think it's time for something more focused and smaller to emerge to tackle the noise level. That this is likely to be many smaller things and that I'd miss part of the debate is kinda fine, I'm fine with missing some stuff if the standard on the bits I'm more strongly interested in can be raised and I don't miss that stuff.

Alex39172• 15 years ago

When HN first started it was kind of like r/truereddit but with an emphasis on startups. HN today has almost nothing in common with this. That's not necessarily a bad thing, but I don't think it's correct to say that the quality level is just an oscillation.

_delirium4• 15 years ago

I guess I don't really agree; HN of several years ago typically had much briefer and less "weighty" articles and discussion than r/truereddit, and a lot more offhand stuff, like the typical techcrunch stories, and "here's how you do a thing in vim" type quick-tip articles. Certainly some good, in-depth technical articles and discussions, but that's the case now too. I've been digging through archives to try to remind myself of whether I had missed some sort of golden age (I was a lurker then, and considered it an interesting but not amazing site), and I still don't really see the golden-ageness, even in retrospect. Nothing like the Kuro5hin golden age, anyway. ;-)

Aegean• 15 years ago

I don't think this answer holds true anymore, i.e. the newbies who are a year old would claim comment quality is going low. I think it now has become the norm. Why not just aim for a standard near the top quality submissions?

cmars2322• 15 years ago

Nice chart. How did you make it?

jdap• 15 years ago

It'd be nice to put in another curve for 'value of HN comments in shaping startups.'

Do those peaks feed periods of frenzied success (inspiring people to do better stuff faster)?

Or do they lag periods of success (pearls of wisdom come out when people are past the sweat-soaked white knuckle phase)?

Or is there some other correlation?

aDemoUzer• 15 years ago

Part of the problem is the basic UI look, hence I am working on a new UI for it: http://peri.me/2B1A/

newguy889• 15 years ago

Political stories simply need to be killed with prejudice.

alienreborn• 15 years ago

One solution can be to give read-only access to new users and charge a very low one time fees for upvoting, submitting and commenting. Only people who are serious about contributing will pay for it.

ck2• 15 years ago

Remove points from users (keep on submissions/replies for positioning).

Problem solved by changing motivations/behaviors.

adrianwaj2• 15 years ago

Does anyone here think crowd-sourcing due-dilligence on startups would be a good idea?

pclark• 15 years ago

Come now, I can't be the only one that finds the Hacker News quality "good to great"?

If Hacker News is about hackers in a startup sense, it's good that the front page has everything from: Movies being in decline - Ruby concurrency explained - A torrent meta search engine - Windows 7 SP1 launch - iPad2 being unveiled.

There are far more elements to hacking than programming, just as there are far more elements to startups than programming. And I dig that Hacker News is so varied.

I think there is a vocal minority of people that get irritated by bicycle shed debates (+1 from me to allow collapsing comment threads on my machine) or people wanting to only read about programming or hacking - the latter of which is laughable because I am pretty sure you'd be sick of Hacker News if it was 100% a specific topic (I have some scars in the field of sorting content users will enjoy...)

Guess what: there are millions of non technical silent people on the internet, and a huge amount of those people visit Hacker News every day - and love this destination. The amount of random non computer scientists I meet in Cambridge that love Hacker News is staggering.

jsmcgd• 15 years ago

I now check 'new' articles more often. There are a lot of gems in there that are not voted up.

adrianwaj2• 15 years ago

I agree with Alex, I think the success of HN has led to an overflow of people dissatisfied with HN who have made HN a success in the first place (and some with YC itself.) I posted two ideas that could be of interest to such people here:

http://news.ycombinator.com/item?id=2253752

cfontes• 15 years ago

Every single community driven website on earth have being thru this kind of cycle the only way to stop this is to start a new one that will end getting into this cycle again. It's a matter of the amount of people using it, on the beginning only very early adopters and people passionated about that specific topic (here tech startups)are in, as years go by more people that have more interests start to join and post things they think is good, and then the topic changes to a more general subject.

I like this community and I think the quality will always fluctuate but the most of it will always be very good content for people in a hurry.

Thanks for all of you who help this place being nice.

nbashaw• 15 years ago

It's important to separate concerns, and I think there are three main things people are complaining about:

1) quality of submissions 2) quality of comments 3) quality of community

It's the second and third that I think have declined. It's not because the people are any less smart, it's that there are just too many of them. It becomes difficult to keep a mental model of everyone in your head, so you start seeing everything as disembodied text, rather than human beings speaking to one another, with a history of shared experiences.

IMO, this is a solvable problem. You can use avatars and display our locations next to our comments, or even just make our names a little bigger. Anything to humanize the conversations.

davidhollander• 15 years ago

Simplest solution

Limit the number of links submitted per account per day to 1.

Why

Prevents spammers and karmafarmers from submitting the entire TechCrunch\Wired back-catalog at a rate of 25+ a day.

Further Analysis

Increasing the scarcity of a resource (link submission ability) will increase the value of items it is traded for (links).

HN already gets the independent code submissions people want. They just die an early death on the new page due to overcrowding by webzines\newspapers with builtin linkbait titles. This reduces the rate of dropoff for independent news.

RiderOfGiraffes• 15 years ago

In response to some of the points being raised in the thread, and to provide some sorely needed data, here are some snapshots of the HN "newest" page taken since Feb 2009:

http://news.ycombinator.com/item?id=2254397

cmars2322• 15 years ago

There is no hacker news problem if you realize that all sites up to this point in internet history are either heavily curated for quality and limited in scope or self-moderated with a one-dimensional, imperfect karma game, and thus chaotic and ephemeral.

Such is the nature of suchness until someone figures out a better game that more properly engages human nature.

Crying over the demise of HN is like crying over a naive hill-climbing algorithm when it gets stuck.

howlingmime• 15 years ago

This has probably been said before (and/or above), but perhaps each article should be tagged and users should subscribe to only those tags of interest to them. In addition, a social component to HN would be useful -- for instance, allow me to recommend a story for someone or for stories of interest to my friends to be ranked above the norm. Our friends make great filters!

protomyth• 15 years ago

Almost all sites that have comments and user moderation concentrate on the comments and up voting / down voting them. Normally, when dealing with people, I don't remember the individual quote that made me think they were brilliant / a troll, I wrap that up into my sense of them. If I'm flipping channels and see someone who has struck me as a brilliant commentator, I stop based on the visual cue of their face. Names are kind of hard (is this the guy who called me a $%$% or was he the one who really knows python?).

I guess I wonder if the same thing keeps happening on "comment moderation" sites, isn't it time to look at the ways your view could be based on your (not the group's) opinion of your fellow commentors? I don't have a technical suggestion, but I will probably think a lot on it.

Solving The Hacker News Problem

Related Stories