A steep rise of Hacker News in Google rankings

https://jonathanpagel.com/the-steep-rise-of-hackernews-in-google-rankings/

by jcmp • 2 years ago

356 357 2 years ago

kgbcia• 2 years ago

We need more sites like hackernews. X /twitter used to be that , but it's now overwhelmed with bots and SEO. The more boring you make a site (no colors, no images, no links) the better the defense against spammers.

beachy3• 2 years ago

Downvoted but true.

HN's pedestrian design makes for a much better experience.

- no flair for usernames - means people concentrate on the message not the messenger

- no visible karma for anyone but the top few - means people don't spend (as much) time karma whoring

- limited formatting, no images or video - increases the value and import of the written word

- no sharing, or @user referencing - means posts live and die more by their merit rather than brigading or other shenanigans

mikestew• 2 years ago

no visible karma for anyone but the top few

It took one click to see that user abraae has 6671, as of this writing. Or did I miss the point in dramatic fashion?

lnauta• 2 years ago

On top of that no sharing/@ing: no notifications for responses (there's probably an addon for that if you want it).

At first I thought I wouldn't like it, because I tend to post on subjects I'm knowledgeable and want to do my best there. But now I'm so glad to not get the anxiety and rush of the back and forth in heated discussions.

Peace and quiet.

toomuchtodo• 2 years ago

Optimized for thoughtful discussion over engagement.

ho42• 2 years ago

I would say this site user design wise is about the same as reddit, so I don't agree with any of your points. What makes it higher quality is that it has a niche theme (hackery) and it makes it less popular. As in general online, the less popular a resource is, the higher quality communication it has. You can find the same quality of conversation in low population subreddits too, despite the aforementioned design.

rldjbpin3• 2 years ago

with the experimental spirit of the site, the barebones look is enough for the "mvp". but for most of the web users today, it is not quite as engaging to use.

most of the readers here don't mind reading text and face a wall of text daily, so it works out here.

bobthepanda• 2 years ago

bots and SEO have really destroyed the internet. and now that we have AI to further streamline the production of BS, I feel like the Internet is just going to become even more of a quality content wasteland.

theturtletalks• 2 years ago

Goodharth’s Law destroyed the internet. The constant game of chess between Google and SEO marketers has the turned the whole search product to crap.

It won’t improve since when Google makes a change, SEO marketers adapt. The websites that actually provide value and don’t really care about SEO suffer as well as the users looking for that exact information.

1270018080• 2 years ago

I'm just waiting for some kind of real person yet anonymous protocol that gets introduced in the next era of forums.

Devasta• 2 years ago

I fully believe that, especially with the rise of AI models, the future of the internet is going to be small enclaves of a few thousand people on invite only message boards. Anything else is just going to be far and away too much effort for anyone to maintain, especially when advertisers twig that their ads are mostly being shown to bots.

I just don't see how anything else could be sustainable.

vladms• 2 years ago

Accessibility to the internet to everybody in the world destroyed the hacker haven that the Internet once was.

But I don't think it's bad. Hackers/smart people "locked in the basement' or talking only with their friends in their bubble is not ideal. There are a lot of people out there, with their own opinions, ideas and understanding (or lack of) of the world. Internet just converges towards the average human being. If we want a better internet, hope some smart people will put some effort to make the "average person" in the world wiser, rather than blame the SEO and bots...

sph2• 2 years ago

https://en.m.wikipedia.org/wiki/Dead_Internet_theory

LLMs are only a deathblow, albeit a massive one, to a trend that's been 10 years in the making.

I feel it's time for another small internet for us, with blackjack and dancers. But this time let's agree not to make it friendly and accessible for everyone, alright?

Arrath2• 2 years ago

I swear my girlfriend relishes in reading me the entire, 300 words of SEO bait product listings on Amazon. Babe, I'm begging you please, you can stop at "xl dog bed" I don't need to hear the rest that goes "fluffy for best friend comfort for large dogs pitbull great Dane german Shepard..."

fsflover• 2 years ago

Mastodon: https://news.ycombinator.com/item?id=35224356

nonrandomstring• 2 years ago

HN requires no javascript and is accessible through Tor without issues. It feels right for those of us who are usually marginalised by a wish for better security.

Feels well moderated too, by people who care about the place, and that's the core of a sustainable community. Care really matters.

ametrau2• 2 years ago

Exactly right. Whereas with Reddit you get the feeling the moderators have a very unhealthy relationship with the site.

jseliger• 2 years ago

You can still (basically) create your own HN / Twitter using RSS—my favorite reader is NetNewsWire, but there are others—particularly with the rise of Substack. I have a couple hundred feeds in mine, and it's great.

On my own site, I routinely post lists of links to interesting articles. You don't have to rely on Twitter or other highly botted sources.

That most people do is itself revealing.

Retr0id• 2 years ago

The more interesting/useful part of HN is the comment section, not necessarily the articles themselves.

spartanatreyu2• 2 years ago

Slight disagree.

"More boring" is a defence against spammers, but not the only the one.

There's also: "Take commercial incentives of the running company out of the picture."

The most prominent example of that is Mastodon. It's software is opensourced, and it's most popular server instance: https://mastodon.social is run by a gGmbH non-profit. (It's hosting company runs it as a non-profit charity for the social good)

Since it's developed in the open without financial incentives muddying up the experience, no advertisements are added and there aren't any algorithmic rankings to be gamed.

And since it's also based on open source, it's easy to share content from other server instances (it's all ActivityPub protocol underneath), and it's also easy to block (defederate) server instances with trolls and other problematic users.

------------------------------

It's like how twitter was at the start, but better.

If you want to try it out, you can make an account on any server then follow some developers in your languages/libraries/tooling of your choice.

You'll also see what those maintainers are discussing in the open and get an idea of how your languages/libraries/tools are going to evolve in the next version, or even participate in their evolution.

jerbear43283• 2 years ago

Another important factor, imo, is size. Quality on platforms like Twitter, Reddit, HN, Mastodon is inversely proportional to their size. If a platform gets big enough, regardless of its motives or polish, there will be more incentive to game it.

Platforms like HN and Mastodon are great because they are small. They cater more towards a smaller, more technical community, which it isn't worth it to game with spam or whatnot because they're small and more aware of this kind of manipulation. Smaller "gems" in bigger platforms (think a small, old subreddit) can be good for the same reason.

I guess this advocates more for the small web, which I'm all for, but there's less money in that. I wonder what could practically what incentives could make the web smaller and more useful.

matheusmoreira11• 2 years ago

We need more old school forums. Decentralized communities focused on their respective niches.

daemonologist• 2 years ago

I think a large part of the reason hackernews is good is that operator's incentives are more closely aligned to the desires of users than average. Ycombinator benefits if HN is the best place to discuss the creation of technology/software because it boosts their brand and gives their companies a communication and recruiting advantage; most other platforms are primarily interested in ad revenue and are only incentivized to provide a good experience to users as a factor in that equation.

Defense against spammers/bots is a tough problem though. Having great moderators and users who are savvy and intolerant of bad content probably helps but I think it would only go so far on a large site. HN might benefit from the relatively narrow appeal of its content in that regard.

wnevets2• 2 years ago

We need more sites like hackernews

HN is heavily moderated (censored) and people love to complain when that happens on sites like twitter.

AuryGlenz2• 2 years ago

It’s moderated for quality, not opinions. That’s what people complain about (and I haven’t seen done elsewhere).

ctrw2• 2 years ago

Reddit was the more popular hn replacement. Twitter has too weird a form factor to ever allow the possibly of anything but hot takes.

trashtester• 2 years ago

In a world where AI can impersonate people to virtual perfection, how would one even KNOW who to invite?

I think proof of identity may soon become the only way to keep AI powered bots from trying to manipulate every forum out there.

And for those who cherish the ability to be anonymous, there is probably a market for a trusted middle-man/site that can verify that an account is being created by a biological person, but that doesn't provide the actual identity of that person to the site where he/she is setting up a new account. Kind of like CA providers do for ssl domains.

bryanrasmussen2• 2 years ago

so the theory is that now there will be a lot of bots taking over HN and posting stuff that they want to rank up, destroying the value of the site? What's the defense there?

jcmp5• 2 years ago

Bit afraid about that. Honestly it is suprising how spam free HN is, so maybe they have some good anti spam system in place. Also i think it would make sense to set all outside links to "nofollow" so google ingores them and therefore it is less interesting for SEO.

MrBlueIncognito• 2 years ago

That would also eliminate any positive effect that HN has on Google search results.

sebzim4500• 2 years ago

I wonder if nofollow actually does anything nowadays. I feel like the rules that used to be enforced have been replaced with hacky 'AI' that have been hacked together by thousands of SWEs trying to improve some metric for their promotion packet.

spacebacon• 2 years ago

It gets its fair share of spam but the platform is very effective at moderation. Show dead and scroll new for examples.

TheAceOfHearts• 2 years ago

There's definitely a bit of influence / perception manipulation on HN. A few years back I heard a story that a tech company would monitor HN for certain keywords, and if their product or category was ever brought up or mentioned multiple developers would always show up to engage on the topic. This isn't quite spamming or cheating the system, but it's a very effective tactic for shifting public perception.

ldjkfkdsjnv• 2 years ago

Theres alot more undervocer product pumping than you would think

xanderlewis3• 2 years ago

Once you get enough karma (I can’t remember what the number is) you’ll be able to see dead submissions. If you look at new you’ll see that there are spam posts coming in every couple of minutes or so. They’re just very effectively detected, as you suspected.

whiterknight2• 2 years ago

You dont recognize the spam because the bad ones are removed. Good marketing targets its audience, and influences people who are not associated to repeat its talking points.

jasonjmcghee• 2 years ago

People are starting to copy/paste from ChatGPT / equivalent. It's not spam, but it just says nothing and dilutes the conversation.

fluidcruft• 2 years ago

Wouldn't the defense be to delist and starve/block the google crawlers? Gaming google rankings is a waste of time if it's not there.

EarthLaunch• 2 years ago

The new way to build communities: SE-anti-O. Trying to rank as low as possible. Things will flip and sites will try to be uncrawlable to Google. I guess it's already happened with Discord.

CableNinja• 2 years ago

You just gave me an idea for a fun project (like i need more projects!). I use openresty for my webserver, which has lua already built in. Might be fun to see if i can just hog up a crawler instance indefinitely by generating fake pages and links that just go back to make a new fake page and link and takes the crawler in circles that it thinks are unique

pixl973• 2 years ago

Over the past few weeks I've seen more crypto scam posts than ever. While they quickly become dead, I'm not sure if they are coming in a faster rate than in the past or what.

dang15• 2 years ago

I've seen that too. Usually these things come in waves and once the spammers figure out it's not working, they go away. Then new spammers show up.

factormeta• 2 years ago

HN can always change their robot.txt and prevent google from crawling. Many of copy left friends do that to their blog and site to prevent google/bing from indexing them.

sitkack2• 2 years ago

This is really counter productive towards furthering the cause.

xp843• 2 years ago

Just so you know, that doesn’t stop Google from indexing you anymore. It only stops Googlebot from crawling. In other words it’s strictly a place to deny permission or set limits on a technical level. If you do this they will still have what they already crawled and if there are links to it on the Internet they will keep it in the index.

If you don’t want something to be in Google on a policy level, you now have to add the noindex meta tag on every page.

armchairhacker• 2 years ago

Invite only.

A site is spam-proof if it makes it hard enough to get an invitation. The relatively few spammers who slip through can be warned and/or banned.

Unfortunately, we may not be invited.

throw1092012• 2 years ago

This is what Lobsters does, and it's pretty effective (although the site has other, much worse problems that outweigh that one feature).

withinboredom10• 2 years ago

Heh, that doesn’t stop spammers. It only takes one spammer getting invited before they start inviting their spammer “friends”. Before you know it, you have thousands of dormant accounts just waiting to spam you. It makes it harder, sure, but that’s not an effective solution by itself. In fact, it leads to a false sense of security.

lelanthran3• 2 years ago

A site is spam-proof if it makes it hard enough to get an invitation.

Yeah, but then it's also invisible to search engine users, in which case no one will want to spam the site for SEO anyway.

matheusmoreira11• 2 years ago

Prohibition of new user registrations.

saiya-jin• 2 years ago

It doesn't have to be prohibition which over time would harm HN badly, just slower and delayed approach of earning good access. Or show age and karma of users commenting next to their nick, some scam patterns will become obvious.

precompute• 2 years ago

It's already a thing. With showdead, you can see the corpses of tens of marketing posts at the bottom of a submission with a high score.

calderknight2• 2 years ago

worldcoin integration

j-a-a-p• 2 years ago

Cut off the infected limb and block the site to Google entirely. Or split HN in an active and archived branch with an upvote threshold, the latter is made accessible by Google.

sitkack2• 2 years ago

Won’t that make everyone with a grandfathered account an AI Boomer? The value of selling HN alts just went way up!

OrderlyTiamat2• 2 years ago

There is already an excellent search site for hn [0], so if appearing in google turns out to cause problems, for many users nothing much will change. (I already use that one over google with "site:.." in the query- it's just a good search experience)

[0] https://hn.algolia.com/

renegat0x0• 2 years ago

I really dig what HN is doing with karma, and stuff.

I really like what lobsters did with invitation tree https://lobste.rs/about#invitations

monero-xmr• 2 years ago

Maybe it’s because HN is well moderated and open to automated scraping, unlike other systems that are being locked down (reddit, Twitter) so the user generated content is now significantly more valuable to search engines.

m463• 2 years ago

doesn't chrome track your browsing history? Could be that people spend time on certain sites, affects what google trusts and/or thinks is popular and ieeds into the ranking for searches.

who knows.

m2mdas2• 2 years ago

Eternal september more accelerated.

tcldr• 2 years ago

Dear Google, we were only adding ‘site:reddit.com’ or ‘site:news.ycombinator.com’ to search queries so we could get to opinions that weren’t being manipulated by SEO fiddlers. What’s our alternative now?

teruakohatu2• 2 years ago

Spammers have been heavily targeting reddit for years and site:reddit.com is still useful.

Why Google have not bought reddit, I don't know (beyond moderation issues, but AMZN made it work with twitch).

candiddevmike• 2 years ago

I've become really skeptical of Reddit comments around products for this reason. Searching "best X site:reddit.com" and going off the top comment recommendation seems really sketchy when that top comment is only 5-10 points.

Maybe I'm just really paranoid these days, but I would bet looking at searches with Reddit in them and creating threads or commenting on old ones and paying for up votes is probably lucrative.

arccy• 2 years ago

why would they buy a cesspool of bots and memes? google is for finding content, not hosting content (unless you want it shut down)

tmpz22• 2 years ago

I don't think Reddit has ever been a clean acquisition - either because they've raised at high valuations many times or because of an undesirable content/moderation problem.

shagie• 2 years ago

Do you believe that Google would have been a better custodian of Reddit than say Deja News ( https://www.wired.com/2001/02/google-buys-deja-archive/ ) aka Google Groups ( https://www.pcmag.com/news/end-of-an-era-google-groups-to-dr... )?

overstay89302• 2 years ago

Google hasn’t bought Reddit because it’s literally only downsides. Bad PR, low profit margin, etc…

I mean does anyone actually like Reddit anymore? Front page is almost entirely bots reposing and reusing the same comments that have been used for years.

ikiris• 2 years ago

Nothing would ruin reddit faster than being a google "bet"

randmeerkat• 2 years ago

Why Google have not bought reddit, I don't know…

My guess is that too many people at Google are on Reddit and didn’t want to see it go to the Google graveyard the day after acquisition.

Hnrobert42• 2 years ago

This scares me. What happens when HN becomes the next spammer target? Where do we all migrate?

andsoitis• 2 years ago

but AMZN made it work with twitch).

I don't know that the Twitch acquisition was smart. What was the strategic reason to acquire?

Amazon announced cutting 35% of Twitch staff (500) in January to stem losses after two rounds of layoffs last year.

umvi• 2 years ago

I've started using smaller (ish) YouTubers instead of reddit when it comes to finding best in class products. For example, this channel is phenomenal: https://www.youtube.com/channel/UC2rzsm1Qi6N1X-wuOg_p0Ng

ametrau2• 2 years ago

Looks better than most but the only reason it’s video is to show you ads. It’s the wrong format and it’s awkward.

CapitalistCartr• 2 years ago

Project Farm is a treasure.

iancmceachern3• 2 years ago

Yes, project farm is a treasure

AH4oFVbPT4f8• 2 years ago

I sub to very few channels but he's one of my favorite along with CompanyMan videos

w-ll2• 2 years ago

I need a `age:1year` or something, almost everytime I search something tech related i generealy dont want some StackOverflow from 2014.

And dont get me started on the changes they made if you search a product or something buyable. its takes like 4 clicks to go to the shop page and none of the open-in-tab methods work

PinkMilkshake• 2 years ago

   before/after:<year>

Works on Google and Youtube

unclebucknasty• 2 years ago

I think that entire "Hidden Gem" update is being spun by Google as a positive "opportunity" to bring in more useful content for users, when it is actually a defensive maneuver against the absolute gaming of their algo that has led to an insane deterioration in their results quality. Even simple queries now routinely return mountains of obvious SE spam.

Combine this with Google now placing only sponsored content on damn near the full first SERP for some terms and it has become less useful by an order of magnitude (I meticulously calculated this figure).

And, this degradation at the same time ChatGPT has come on the scene. I know I personally bypass Google altogether now more and more frequently in favor of ChatGPT. Wonder how many people do the same and whether there is a whiff of desperation at the Big G.

aaron695• 2 years ago

I had a weird experience today. I saw the forum Hackernews in my Google results

Click the 3 dots (kebab) beside individual search results and see if it was -

"Personalised for you"

This is the most likely answer.

[Edit] "where was gta made" does come back as "Not Personalised" in incognito mode and I see OP is in SEO.

I get a lot of results like this but all OPs examples seem "Not Personalised"

Scubabear68• 2 years ago

HN deserves to be at or near the top of search results because the quality of discourse is so high here, and on a very wide range of topics.

I have noticed an increase in errors here in the past few months though, I wonder if this is the cause.

chipweinberger2• 2 years ago

I think the only defense for increased spam is accounts will need to be tied to real people through some other identification.

Sucks, but is there an alternative?

I’m open to the idea of no new accounts if it got bad enough.

I recently switched to more actively using my account that has my name as the username, anticipating the growing problem of spam detection, and trying to make it more clear this account is not spam.

pixl973• 2 years ago

Which identification is that?

Also, identification is not something that is revocable. If someone steals and abuses your identity (though no fault of your own?), are you now banned from the internet?

chipweinberger2• 2 years ago

those are solvable problems, but no one is solving them today.

there’s no identity system today that I would want HN to use.

qingcharles• 2 years ago

Also, ID is not available to everyone. I can't get ID right now due to being an illegal immigrant.

em-bee• 2 years ago

how about submissions only from accounts that are no longer green (if that's not the case already), and possibly also have a positive comment posting history. although with AI it will be possible to make bots that write seemingly coherent and positive comments, so unfortunately this will continue to be an arms race.

maybe submissions only after reaching a certain number of points. that would reduce the number of people that can submit though.

another alternative would be to clearly identify which submissions come from posters with a high number of points. that would allow anyone to submit but make it easy to give posts by weak accounts a greater scrutiny.

mech4222• 2 years ago

that sounds more list lobste.rs ...

you have to be 'approved' to post a topic, which you get by commenting on existing topics over time. TBH, I've never even bothered opening an account as I find it relaxing to be in 'read only' mode :-)

vunderba• 2 years ago

I was part of a HN-style tech site many years back that used an interesting system where new submissions would actually require you to spend your karma points. Points would accumulate based on your standard karma voting for comment history, but a small amount would also accrue for free on each day you logged in.

ahmedfromtunis• 2 years ago

What's worrying here also is Google's willingness to use a random comment as the source for its snippets.

I understand that the average HN commenter puts more effort into their comments, and their veracity, than the average internet user elsewhere, but still.

(For me, the top "result" for Monaco in italian is the Google Translator widget with Collins Dictionary, Wikipedia and Quora the top 3 links. HN is 4 or 5 links below.)

Oh, and yes, let's hope spammers don't overwhelm dang!

tayo42• 2 years ago

What happens when hn outgrows it's current self? Even now, the front page is pretty fast paced. Things get burried pretty quick. Sometimes I see an interesting post and comments and think to my self I want to come back later when I have more time but it'll be a few pages off the front by then.

dang15• 2 years ago

Curiously, the number of submissions per day hasn't grown much since 2011. Everything else on HN grows more or less linearly, but not that.

pknerd• 2 years ago

Is it not called "Parasite SEO"[1]?

[1] https://www.kiwop.com/en/blog/parasite-seo-what-is-it-and-ho....

DiscourseFan2• 2 years ago

I started using this website after downloading brave, and I typed "news" into the search bar and it automatically directed me here. I was instantly fascinated by the diversity of articles and high quality discourse, and so I've stayed for way longer than I thought. But, yes, unfortunately, as with Reddit and Xitter, one day HN with suffer the fate of popular social media, probably not too long from now. Social control is the name of the game, and you can't win without monopolizing. I only hope Dang can hold down the fort for as long as possible; HN will be the Masada of the internet. (Well, I suppose there is still 4chan, but that's not going anywhere. 4chan will probably be around long after the death of every other social media website.)

modeless• 2 years ago

If you're new you may not know this but the meme that HN is turning into Reddit is at least 17 years old, and has been the last entry in the Hacker News Guidelines since forever: https://news.ycombinator.com/newsguidelines.html

DiscourseFan2• 2 years ago

Unfortunately, the statistics bear out differently this time.

sherburt3• 2 years ago

A simple way you could stop the enshitification is if you stopped it from becoming too popular in the first place, and a way you could do that is by throttling the performance of the web site to only be able to handle x amount of requests/minute.

tacocataco4• 2 years ago

Anyone know what dang's plans are for post retirement moderation of hackernews?

marbartolome• 2 years ago

4chan will probably be around long after the death of every other social media website

4chan: the roaches of the Internet :)

fennecbutt• 2 years ago

Eh, I remember what it was like back in the day when it was spicy and fun, but ever since moot left and then sold it, it's just not the same.

I haven't really bothered going back, I'd like to say I've grown up since being like 14 and getting pretty accustomed to seeing some pretty terrible stuff on there.

throwawa14223• 2 years ago

Sadly the broader the topics gets on hacker news the less relevant searching site:news.ycombinator.com becomes. This isn't a value judgement on the articles a lot of them are very interesting, but more narrow had some value for finding things.

lobito14• 2 years ago

Google knows your preferences and tailors your search results accordingly. You are The hidden gem. Use anti tracking and privacy tools.

xnx• 2 years ago

I'm pretty suspicious of any 3rd party seo tools. Only true data comes from Google Search Console or hacker news logs.

tussa2• 2 years ago

I often find myself searching on HN, then Reddit (via google) and then good old sad google, in that order.

When Google arrived, it had the solution people were desperately waiting for. It was pretty much everything we wanted. And it even "wasn't evil!"

Now it feels like people at Google are just making sure they'll qualify for their annual bonus with total disregard to what happens to the company.

I forecast that in the next 1-3 years, we'll see another company steal the search market, just like Google it back in the day. But Google, don't worry, they'll "just be a search company".

vik02• 2 years ago

I forecast that in the next 1-3 years, we'll see another company steal the search market, just like Google it back in the day.

Really? Do you think this company exists today?

calderknight2• 2 years ago

I would guess the new chatbots have already stolen a bit of the market.

tussa2• 2 years ago

We'd know about them if they existed as a company today.

Remember that what became Google was Larry's and Sergey's PhD project that no one wanted to buy.

rixthefox• 2 years ago

* huge asterisk since I pay to use it and feel the need to disclose that upfront.

Kagi is a very strong and viable candidate IMHO. The search is actually useful and I’ve noticed on a few occasions that my search was “too focused” and yielded “no results found” and I had to rethink my query to better find what I was looking for. Google on the other hand would have spared no opportunity to spam my results with ads even if it couldn’t find what I was looking for. Anything to put more ads in my face.

figassis• 2 years ago

I've always thought it was strange that HN never appeared in search results. I mean content here has a very high SNR and seems to me that it checks all of Google's SEO boxes. So I always assumed not showing up was intentional, as in dang has been delisting it on purpose and blocking bots. Now that it's showing up, I kind of want it to go back the way it was. The only reason I read HN so much is because I have given it a high degree of trust, that even though I don't know most commenters, I can easily reason about content, find the sources, discover amazing tools and read from founders directly. I really, really do not want to worry about whether the front page is now ads.

culi• 2 years ago

The frontpage has always been ads. That's like the entire point of this website...

I mean just look at the domain name

jart• 2 years ago

Ruh roh. Google is finally fixing the search ranking problem, good and hard. Be careful what you wish for.

CamelCaseName• 2 years ago

I suspect this is because when they rolled out their Perspectives \ Discussions tab, Reddit completely dominated the results. One article I read said they appeared in 97.2% of results.

Qem• 2 years ago

It seems with the recent wave of layoffs people now have more free time to be on Hacker News.

65• 2 years ago

Next Google should boost the rankings of forums. And we can all go back to forums. And the internet can be good again.

mianos• 2 years ago

The weird thing is I just saw a few crypto scammers posting on other threads. Sucks to be popular.

akdaadensb2• 2 years ago

This is karma for us originally starting the fire about typing reddit in our Google searches (I believe it's fair to say this was discovered independently by many technically proficient people over the course of the preceding half decade), but we definitely had a hand in publicizing it.

I'm curious what the end result will be but will be disappointed if the quality of HN decreases to be more in line with Reddit et al.

What makes HackerNews different from Reddit is that people I actually respect frequent here, real players at the game of life and whatnot (whatever that means), and I can't say the same about Reddit. If it gets too popular I can see that being jeopardized.

baq5• 2 years ago

The sad thing is those people most definitely have been on Reddit, but the enshittification hit too hard and most of them just quit.

1vuio0pswjnm72• 2 years ago

"It is also noteworthy what it HN ranking for:

A comment is the top result for "where was gta made"."

Not for me. Wikipedia is #1. I retrieved 100 results. None of them were HN comments.

"A comment from desdiv is ranking #1 for the question "Monaco in italian""

For me Wikipedia was again #1. HN comment was #2.

"Another article that ranks well for several keywords like "ruined my life" is GRAHAM PANTHER's article, and HN also ranks well for brand-related KW, such as every question about the cost of youtube tv, where Google favours the post from thunderbong."

For the search "cost of youtube tv", NerdWallet ranked #1. HN comment was #10.

It is baffling to me that people, who certainly know that results for the same query can vary according to a number of factors, e.g., geolocation, pretend every computer user performing some Google search query is getting identical results.

Ideally that is how search should work. But Google web search is not ideal. Far from it. It is advertising company search. The goal is not to return results in a consistent, transparent manner according to a relatively simple, well-known algorithm. It is to sell advertising services. The goal is not truth. It's money.

skybrian• 2 years ago

Why would you even want the same results everywhere? Local results will be better for many queries.

jcmp5• 2 years ago

You are right ofcourse, results variey a lot. The results in the post where sourced from a third party tool for the UK and the US so kind of the results you would get in inkognito mode.

1vuio0pswjnm72• 2 years ago

"Local search" when using Google is not optional. The user cannot choose a region when searching www.google.com. The advertising company chooses the region. Why. For the user of course. The company is a trillion dollar charity run by volunteers working on users' behalf for "the good of the web". Right.

Does local search benefit sales of advertising services. Proponents of so-called "tech" companies will always try to justify the choices of the so-called "tech" company^1 as being for the benefit of the user. It could be half true.

To use DNS as an analaogy, EDNS0 client subnet (ECS) was another one of Google's magnificient "improvements" to the internet. Google fans would argue that ECS is 100% for the benefit of users. However it just so happens to be useful for advertising.^2 What a fortuitous coincidence.

1. The user does not choose. Choice is removed.

2. https://hacker-news.firebaseio.com/v0/item/13993698.json

bimguy• 2 years ago

For years I had no idea of this forum, no Apple technicians I worked with ever mentioned this site. Rixstep.com linked this forum (in 2016) in one of their articles and that's how I found it. Definitely felt like being given a gift. HN is/was a true hidden gem.

throw1092012• 2 years ago

@dang I would like to make the argument that the increased attention from Google will exert a downwards pressure in HN quality due to marketers taking advantage of its influence and search ranking to conduct influence operations.

Would you be willing to consider the possibility of delisting the site with robots.txt?

supriyo-biswas3• 2 years ago

Mentions do not work here, you should send an email to hn [at] ycombinator.com.

(BTW, if this is gonna be done, we should also block all AI bots and search engines at the IP address level.)

throw1092012• 2 years ago

(BTW, if this is gonna be done, we should also block all AI bots and search engines at the IP address level.)

That's not possible or necessary.

Not possible, because it simply isn't possible in the general case to differentiate real papers from scrapers, without using device attestation. For an extreme example of this, see the Recap the Law project[1], which gives real human users an extension which scrapes as they browse.

Not necessary, because scrapers for AI training data an entirely separate problem completely unrelated to marketing, and because robots.txt will serve to stop the majority of search engine indexing, which is all that we need. Actual blocking of engines isn't necessary, because all the big ones are well-behaved, and after they stop indexing HN, marketers won't care about HN for the purposes of SEO-related influence campaigns any more.

wodenokoto• 2 years ago

While @‘s aren’t an implemented feature, dang has quite the ability to find them.

sfmike• 2 years ago

Would like to see this argument made. There should also be an argument made that more good contributors would come. In face, more people without the mentality of this post to block and hinder visibility, in effect, would mean this is a net positive for aggregate sentiment.

Erratic6576• 2 years ago

I would like to make the argument that the increased attention from Google will exert an enshittification pressure in code quality due to the sudden appearance and accumulation of trackers, until the web is finally sold to any big publishing company, and, eventually, to an AI training company.

Browsing HN is a relief in times of Surveillance Capitalism

Edit: HN uses google tagmanager and analytics; amplitude and branch.io

dang15• 2 years ago

HN uses google tagmanager and analytics; amplitude and branch.io

We do? No we don't? I don't even know what those are.

piva00• 2 years ago

I don't see any requests to any analytics or Google Tagmanager, actually the only requests I see when opening a post are to the `news.ycombinator.com` domains to fetch the document itself, a CSS file, a JS file, an image and a couple of SVGs, nothing else.

If you see requests outside of that there's something fishy going on in your browser, I believe.

ogurechny• 2 years ago

I think you are trying to build walls around something that isn't entirely yours, both personally and “you-as-the-voice-of-the-community yours”.

The linked blog post actually describes that Google is drowning in ad-driven crap, and has to rely on manually chosen external structures to keep quality tolerable, not that “HN has became cool”.

throw1092012• 2 years ago

I think you are trying to build walls around something that isn't entirely yours, both personally and “you-as-the-voice-of-the-community yours”.

Profiling other users, in addition to being extremely bad form, is a pretty good sign that you don't actually have a valid argument to make - which, reading your comment, you don't.

dang15• 2 years ago

HN has been under downward quality pressure for a long time. Figuring out how to withstand it has been the core idea all along:

HN is an experiment. As a rule, a community site that becomes popular will decline in quality. Our hypothesis is that this is not inevitable—that by making a conscious effort to resist decline, we can keep it from happening. - https://news.ycombinator.com/newswelcome.html

pg wrote that over 15 years ago. I've been saying for (a mere) 10 years that we're trying to stave off the arrow of internet entropy: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... – not doable forever, no doubt, but doable for a while if we expend enough energy.

Based on experience so far, the way to do this is through a combination of a dedicated community, software that does what software can, and moderation to bump the system out of its failure modes.

Trying to hide from the outside world is mostly not the way. pg used to do tricks like Erlang Day on occasion, but having that as the main strategy would be like trying to avoid infection by never going outside. Far better is to have a robust immune system, if possible. Trying to avoid (resistible) pathogens weakens the immune system, and hiding HN from new users is a path to decrepitude. The latter is probably a greater threat IMO, and too easy for established users to discount.

Spam, and its cousins like content marketing, could kill HN if it became orders of magnitude greater—but from my perspective, it isn't the hardest problem on HN. That's because of the dedicated community, which flags these things, reports them when they escape in the wild, and is vigilant about quality. Without such a community, HN would have died long ago.

By far the harder problem, from my perspective, is low-quality comments, and I don't mean by bad actors—the community is pretty good about flagging and reporting those; I mean lame and/or mean comments by otherwise good users who don't intend to and don't realize they're doing that. There's an unholy dynamic between those and the upvote system, so worse comments often get upvoted more than better comments do—often enough to choke the threads with weeds. That's the high-order bit and what I spend more time worrying about—not Cassandra E Oakley and her trading system*, nor the latest startup voting ring and whatnot. If those ever become the high bit, we might be doomed, but we should see it happening long enough in advance to readjust.

p.s. (@dang doesn't work - I only saw this by accident. Well, not by accident because it ended up at the top of the thread)

* https://news.ycombinator.com/item?id=39425371 - unkilled for the occasion

defrost• 2 years ago

p.s. @dang doesn't work

blame those dang programmers!

zamadatix• 2 years ago

Maybe it's not the growth of a community which really causes decline in quality but where the growth comes from. Folks that come because they saw a discussion link in a git message or startup conversation are likely to have very different interactions than folks that come from search engine or social media share.if 100k users showed up tomorrow from the former category would quality here really go down?

I agree it probably doesn't make sense to try to block growth sources though. Just considering the perspective "it's not effective to try to play whack a mole anyways" already seems enough evidence of that alone.

All this reminds me though I've been using the site too much lately and my interaction quality has gone down. Time for another short break :). Anyways

ViktorRay2• 2 years ago

Dang I think you are a really good dude.

So many places on the internet used to be wonderful places of discussion. I remember as a teenager I would come to the internet and be in awe at this system that mankind created to allow for all humans to come together and discuss.

But I’ve noticed since 2015 every single discussion place I used to frequent online has become horrible.

I discovered hacker news only recently and this website seems like the last remaining jewel of the internet that still exists. I think the reason is because of you and the philosophy you use to moderate the website.

Just wanted to express my gratitude to you. Not sure how often people say thank you to you for the work you do but they should say it more often!

lisper4• 2 years ago

There's an unholy dynamic between those and the upvote system, which means that (by default) worse comments get upvoted more than better comments do...

Can you elaborate on this a bit? I don't see why "lame and/or mean comments by otherwise good users who don't intend to and don't realize they're doing that" should have an "unholy dynamic" with the upvote system.

(FWIW, what I have observed is that once a comment becomes established as the top comment in a thread -- and it doesn't take much for that to happen -- it is nearly impossible to dislodge it. That means that getting into a thread early is crucial for getting noticed. I've pretty much stopped commenting on threads that are older than an hour or two because I can be 99.9% certain that whatever I write will never been noticed no matter how good it might be. And FWIW2, the comment I'm responding to is 50 minutes old as I write this.)

supriyo-biswas3• 2 years ago

On that note, Discourse nudges you to do a tutorial on how to use their forum through the private messages from Discobot. I wonder if something like that which shows the kinds of comments that the community needs, along with the upvote limitations I’ve talked about elsewhere and privately messaged you about, may provide a sufficient mitigation to this concern and help users be better contributors in this forum.

Also, stuff like Erlang day isn’t a bad idea: it is also a kind of nudge to remind the community about what kinds of discussions and comments to prioritize.

loveparade• 2 years ago

I disagree with the harder problem. I agree that the problem is low-quality comments, but not the lame or mean kind. When I see comments on a topic I happen to be an expert in it's obvious to me that the majority of them are misinformation or uninformed users. These people don't mean harm, they are legitimately misinformed or inexperienced. There typically are insightful comments as well, and often they are upvoted, but they have become the minority compared to the low quality comments. This ratio has steadily gotten worse over the past years. It's not as bad as reddit, but every pseudonymous community seem to suffer this problem as it's directly tied to trust and identity. There is very little to be gained from comments on HN anymore, I mostly come here for the links now since it's a decent source of news curation.

thinkingemote• 2 years ago

I was browsing the archives of NH the other day and I noticed the quality of most comments to be the pretty much about the same. Similar proportion of off topic or just replying to the title style comments. The length of the comments seemed longer now compared to back in the day. There were a similar number of joke / pun / silly comments which didn't get upvoted. I think there were less lame/mean/snarky comments in the past. I didn't see as much flame.

Would be great to see some actual data analysis.

slekker• 2 years ago

What's "Erlang Day"?

berniedurfee• 2 years ago

Have quotas ever been tried or considered? Like users being allocated a few upvotes per day or week?

I wonder if that would give enough pause for users to consider if a comment is worth their upvote.

throw1092012• 2 years ago

You make very good points, as expected.

However, while I completely agree that spam isn't anywhere close to being a threat to Hacker News, I'm not so sure about subtle, well-executed content marketing/influence campaigns. By design, these are much less overt than mere spam links to a porn website, and there are plenty of places for them to pop up on HN - for instance, in any number of recent threads lamenting the sad state of modern appliances/tools (like vacuum cleaners, e.g. [1] - and it would be extremely difficult to even approximate the scale of the problem.

I'm not advocating hiding from the outside world in the same sense that one might try to intentionally isolate their community to prevent others from joining - just in the sense of avoiding a specific discovery mechanism (Google) that results in extremely strong incentives for content marketing companies to infiltrate and influence. More specifically, I posit that the amount of organic discovery and community growth that we would get would be significantly outweighed by the attention from content marketers - most people that I know discovered HN through blog posts and individual recommendations, and not through search engine results.

Ultimately, of course, it's up to you - but please think about this issue (and particularly the tradeoff of the upsides vs downsides of being highly-ranked in Google results), and look for indicators of content marketing. Hacker News has, by far, the highest quality-size product of any community that I've seen on the internet ever, and I would dearly love for it to stay that way for many decades to come - which I don't think will happen if "Hacker News product recommendations" becomes a currency with supply comparable to GitHub stars[2].

[1] https://news.ycombinator.com/item?id=39238010

[2] https://news.ycombinator.com/item?id=36151140

nolongerthere• 2 years ago

I don’t know enough about web dev to know the performance penalty that might be incurred but, has there been any experimentation with hiding the voting buttons for the first 5-15 seconds after a comment is viewed (scaled up the longer the comment is, to force the user to actually think about the comment before reacting)?

Similarly to how reply is hidden for the first few minutes for hot threads.

ChrisArchitect• 2 years ago

Interesting observation but nothing more than a bit of fearmongering after that. It seems likely that HN content would appear more regularly because it's one of few places left on the net that regular fresh discussion on recent topics is occurring with solid moderation.

Community is hard. And requires moderation. And it's done here with great care. Don't worry about the spam, the fight is real but the problem is also very appreciated on the ground here by Dang and mods as you can tell. Keep it up. And meanwhile you can help by submitting quality content from regularly reliable sources, not upvoting dupes, contributing thoughtful discussion and keeping discussion focused/organized, as many of you do.

coretx• 2 years ago

Since everyone more or less seems to agree on some shape or form of the "problem", what about the following solution;

Create a devils pact with the elite astroturfing people who no one seems to be bothered by or able to identify. Exchange data, and root out the "problem".

If you can't beat them, join them. The lesser evil is still perceived as a better experience after all. Our enemy his enemy is our friend.

MichaelMoser123• 2 years ago

Is it possible that google is now turning to sites that have a strong moderation policy, as a vetting strategy against LLM generated posts? (Maybe November 2022 is turning out to be a new kind of Eternal September for the rest of the Web...)

baq5• 2 years ago

I'd go a bit further and ask what other choice do they have...?

Unless they can make a reasonably effective LLM-generated content detection (sounds tough!) they're faced with a rather inconvenient problem of mining the internet for known human-generated content (while we still agree what 'to know' even means).

Interesting parallels to https://en.wikipedia.org/wiki/Low-background_steel.

thatjoeoverthr• 2 years ago

It’s certainly hard to detect authorship per se, but I expect very easy to detect content people produce using LLMs. You can target affiliate marketing blogs and other SEO garbage, entirely ignoring whether it’s literally written by human hands.

reddalo• 2 years ago

I'd like this definition of "Eternal AI September": all the website that existed before November 2022 and kept posting the same kind of content will be considered safe. Everything else will be vetted to prevent LLM generated content.

Shawnj2• 2 years ago

I bet a lot of text generated content you see online is LLM generated and I'm not sure how exempt HN is from this. Probably more than most but in 2024 it is trivial to generate something with an LLM that could be assumed to be written by a human and post it. Most of the internet pre-November 2022 isn't AI generated (with the exception of one off things like Seychelles Anon, SubredditSimGPT2, etc.)

weinzierl• 2 years ago

Similarly HN is already an important data source for large language model training.

For example, the RefinedWeb paper lists HN as one of only 12 websites that were excluded. From what I understand, it was excluded because it went into the final dataset unvetted. RefinedWeb was used for the Falcon model.

https://arxiv.org/pdf/2306.01116.pdf

3abiton• 2 years ago

Also reddit exodus, aftee their api changes. Don't understimate people's imcline for good open source (or 3rd party clients)

bayindirh• 2 years ago

If your revenue stream depends on your client injecting ads to your users' feed and inability to remove that ad from the said feed unless paid for, you balk at 3rd party clients.

Same is also true if you're implementing half of your moderation at client level.

samuell• 2 years ago

I basically stick 'site:news.ycombinator.com' to most of my searches these days anyways, to the extent I should look for a plugin to have it permanently added (although mostly via DuckDuckGo or Brave Search).

It is a preciously rare gem on today's web.

guappa• 2 years ago

That seems terribly limiting.

_Algernon_2• 2 years ago

Much better signal to noise ratio though.

pynappo• 2 years ago

one can always fall back to regular search queries as needed. i find that searching"<query> reddit" and falling back to "<query>" is usually going to get me an answer faster than the other way around.

jdthedisciple• 2 years ago

Side note: Why do people add

    site:...

to their search query instead of just ... the name of the website?

I do the latter and it works perfectly well, for example "apple vision pro hackernews" or "apple vision pro reddit".

ufo• 2 years ago

On duckduckgo, the apple vision query only returns 3/8 results actually from hackernews.

(I myself prefer using the native HN search / duckduckgo bangs, because it can sort by date and shows the comment count for each thread).

globular-toast• 2 years ago

Note to the author, if reading: the page is marked as German language which makes some browsers prompt to translate from German, despite being in English.

jcmp5• 2 years ago

Thanks, fixed!

TradingPlaces• 2 years ago

Not for nothing, but I was able to raise HN in my Kagi search rankings (and demote others) because I wanted to do it, not because a Googler on the Search Team made a slide deck about HN. https://help.kagi.com/kagi/features/website-info-personalize...

Best $10 I spend every month.

jadodev• 2 years ago

This comment finally pushed me to try out Kagi and WOW. I did not realize how much I was missing out from a search engine - pinning & blocking websites on my search???? It seems so obvious now, but damn is that incredible.

nojvek• 2 years ago

I noticed that Google is ranking Quora very high lately. Which is absurd because the first answer on Quora is usually auto-LLM answered post.

I have to actively exclude Quora to get a better answer.

The enshittification of the Web has truly begun.

A steep rise of Hacker News in Google rankings

Related Stories