"About 67k” sites are banned from submission on HN, the list is kept secret

https://news.ycombinator.com/item?id=37130147

by mnbvcq • 3 years ago

109 169 3 years ago

news.ycombinator.com

momirlan2• 3 years ago

i was surprised myself that hackernews.com is automatically killed. maybe there is a story there...

benatkin2• 3 years ago

Hard to imagine a submission fitting HN guidelines coming from there.

momirlan2• 3 years ago

why ? do tell

anoncow• 3 years ago

We can’t thank dang enough for what he does. Thank you dang.

dnissley• 3 years ago

There should be a dang appreciation day each year

linuxlizard• 3 years ago

Dang right, we should.

wahnfrieden4• 3 years ago

I recommend turning on Show Dead as well, some of the most insightful commenters show up there. It's not a quality filter as much as a d*ng flavor

joshstrange6• 3 years ago

Ehh, I’ve had it on for almost the whole time I’ve been on HN, the comments that are “dead” always have good reason to be in my experience.

Calling them “insightful” does not jive with what I’ve seen at all. It’s often ranting or very clear rule violations.

wahnfrieden4• 3 years ago

There is a wider split between quality. The ones who know they are dead don't expect replies so that does affect their style of posting. Civility is overrated

mh-• 3 years ago

As someone who has also had it on for ~10 years, couple exceptions I'd add:

- New accounts (or long-dormant accounts that suddenly post). I presume there's some kind of filter they're tripping. I try to "vouch" these every time I see them, if they're contributing.

- Good comment, but when I click into their history, they have enough rightfully-flagged-to-dead comments, that it makes sense.

Very occasionally (like <1 in 1000) I'll come across a comment that is dead, and their history makes it look like they've been making good comments without realizing they're dead. I also click vouch on these.

sanderjd3• 3 years ago

I prefer Show Dead as well, but it's definitely not the case that the commenters there are insightful. Every once in a blue moon I see a dead comment that is a shame is dead, but it is very very rare.

wahnfrieden4• 3 years ago

I didn't say that most or many of the dead commenters are quality

kergonath• 3 years ago

I can count on the fingers of one hand the number of insightful dead comments I have read in the past year (and I read all the dead comments I come across). The vast, vast majority of dead posts deserve it.

dredmorbius4• 3 years ago

I subscribe to HN with "showdead" enabled and have for years. Maybe ... once every month or few I find a comment that really shouldn't have been killed. I'll tick the "vouch" link, or for particular head-scratchers, email the mods.

Most often it's new accounts ("greens") which have tripped an automated filter.

Many times though the comments are from banned sockpuppet profiles (repeat registrations by the same person or organisation), according to moderator responses.

(Tracking user history back to a moderator admonishment is ... difficult. In particular, it's not possible to search for comments in response to a specific user. If I could search for "by:dang in-reponse-to:banned_profile" that'd be easier. I suspect mods have better tools.)

two_handfuls• 3 years ago

The list is probably as long as it is because shady sites keep on changing names (for obvious reasons).

theultdev7• 3 years ago

Why is the list kept secret / not published? (it's the same thing)

For one, I wonder if there are certain news sites that are legit but ideals don't align.

Or are they ALL spam sites? Who knows, we can't inspect it.

Fight bad speech with more speech. Spam is the exception, but are they all just spam sites?

abathur2• 3 years ago

https://news.ycombinator.com/item?id=37130234

wnevets2• 3 years ago

Probably so the offending sites don't realize they're being blocked.

theultdev7• 3 years ago

pretty easy to see when they don't get traction / get flagged automatically.

the only thing it really does is prevent others from seeing what's being censored.

input_sh2• 3 years ago

Cause spammers would buy a new domain and do the same thing again, as opposed to plausible denyability.

theultdev7• 3 years ago

they can do that without the list. as soon as you see your post doesn't get traction, make a new domain, or have multiple in the first place.

weinzierl• 3 years ago

Because it invites people to game the system. There are plenty of free lists and they overlap significantly. I'm pretty sure their intersection is also very similar to the core of the HN list.

theultdev7• 3 years ago

Because it invites people to game the system.

Please elaborate, how would they game the list?

They can already see if a submission is throttled/flagged/etc.

kdmccormick• 3 years ago

Somewhat ironically, given how much we talk on this site about FOSS, HN categorically is not FOSS and it does not pretend to be. There are many little behaviors (such as quiet score adjustments) that the admins put in place in order to avoid the Eternal September. If those behaviors were public knowledge, I imagine that they wouldn't really work.

If news like this bothers you, well, then stop giving HN your traffic and go use Lemmy instead :)

thousand_nights• 3 years ago

the only thing that bothers me about hn is that it doesn't let you delete posts so it gets annoying having to rotate accounts as you slowly dox yourself by revealing more and more info

for a social website where so many users are privacy advocates this is surprising to me

yes you can email dang or whatever but that is like having to phone up to cancel an online subscription, dark pattern friction bs

NewJazz• 3 years ago

You also can't block people.

I consider that a pretty important feature for mental health reasons.

wahnfrieden4• 3 years ago

You can't reliably email to request it. Their policy is that if you request your comments to be deleted, they will only delete some minority of them if there is a reason to that they like

nvarsj• 3 years ago

I’ve actually been doxxed from Hn comments and it wasn’t pleasant (although I don’t make a huge effort to hide my identity).

Having said that I think it would be a much different site if comments could be freely removed. So I’m okay with the friction of having to email a human to get your stuff removed.

prometheon1• 3 years ago

HN is a forum, but also a valuable information resource. If it were easy to delete your historical comments, a lot of that information would be lost. If you make a contribution to a Wikipedia page, do you expect to be able to delete it?

I know HN is not Wikipedia, but it is also not like Facebook, in the sense that one can actually learn useful things on HN.

johnnyanmac14• 3 years ago

If news like this bothers you, well, then stop giving HN your traffic and go use Lemmy instead :)

this isn't the breaking point, but this and a lot of other philosophical differences and cultural issues here is putting it close.

I'm just tired of migrating. I especially don't want to think of migrating not even 6 months after I deleted my last Reddit account.

cpach2• 3 years ago

Who’s surprised?

Of course HN is a target for spam.

It’s expected that the list isn’t public and 'dang has explained why: https://news.ycombinator.com/item?id=37130234

Nothing surprising here really.

MBCook• 3 years ago

If you browse new enough you’ll see vaguely topical spam. Every once in a while you see completely off topic spam (ie “$$$ Learn to sell home made soap $$$”).

It’s way too easy to sign up and submit for spam to not be an issue. Of course there is a blacklist of some sort.

calvinmorrison• 3 years ago

(ie “$$$ Learn to sell home made soap $$$”).

Instead it's "How I launched a 21x RoR Environmentally Friendly SaaS using only lye and pig fat"

johnnyanmac14• 3 years ago

I don't like the line of reasoning because "spam abuse" is the boogeyman of the technical internet. You can justify requiring real life credentials to "combat spam", and that is exactly what Meta does and how they spin the PR. Twitter/X just rate limited the top 5 largest site under "spam/scraping concerns".

Spam is annoying but it is like piracy: short of moderating every single piece of user submitted content, it is a game of cat and mouse, and the mouse has infinite time and nigh infinite resources. I don't think sacrificing honesty is a good compromise for this game.

cpach2• 3 years ago

IMO, 'dang & Co does a pretty good job at maintaining hygiene here on HN.

“You can justify requiring real life credentials to ‘combat spam’”

Yes, some would indeed make such an argument. But that’s not what we’re discussing here. We’re talking about a list of banned domains. That’s very far from requiring ID for the members of HN.

branon• 3 years ago

I don't really mind this, it's an open secret that discourse on HN is deejay'd behind the scenes by the mod team and I think it helps set this site above all the rest.

The exact mechanisms by which this moderation is accomplished need not be public, that's the balance between the secret part versus the open part.

Simulacra• 3 years ago

What about the secret list of users shadow/banned?

johnnyanmac14• 3 years ago

yea it probably exists. Much easier to check, though.

joshstrange6• 3 years ago

And we are all better off for it.

I don’t know how many times I have to beat this drum but good communities that people want to spend time in are heavily moderated.

Free speech bros (who don’t actually really believe in it) have yet to create an attractive platform that doesn’t have moderation.

Moderation is key and what some people consider “heavy handed moderation” makes for a nice place to hang out. Without it HN would languish as people with interesting things to add are driven off by the people who think they have a right to say whatever they want anywhere they want to say it.

kyleyeats• 3 years ago

What's a free speech bro?

jjoonathan3• 3 years ago

Me, before I saw forum after forum after forum fall to the Nazi Bar Problem.

(The term is newer than my formative experiences, but it's the best concise explanation of the dynamic I know.)

vkou• 3 years ago

Charitably?

Someone who believes that because we can't tell some kinds of good speech from some kinds of bad speech, we should give up trying.

It makes about as much sense as eating a moldy blue cheese once, enjoying it[1], and deciding that from now on, you won't turn your nose at eating whatever putrescent, rotting garbage is put in front of you, regardless of how sick it makes you.

----

Less charitably?

Someone who can very much tell the difference between good and bad speech, but is cynically promoting bad speech, for selfish reasons.

[1] Despite a general understanding that moldy food is not suitable for eating.

sorokod• 3 years ago

I'll give this a shot:

An XXX bro is someone who aggressively promotes XXX as a common good while actually motivated by promotion of a narrow agenda.

Often associated with individuals that know next to nothing about XXX

Pyramus2• 3 years ago

From what I've learned here on HN someone who thinks that "free speech" is absolute - their right to free speech outshines the personal rights of others. So instead of discussing where my rights end and your rights start, they will argue that no line should be drawn.

In addition, from what I understand, they seem to think that democracy is a consequence of "free speech", not the other way around.

asow927• 3 years ago

Not disagreeing with you necessarily, although, how do we know we're better off for it if we don't know what's on the list?

nickthegreek4• 3 years ago

You don't and they are under no obligation to provide this list. It is not clear to me that providing the list would be constructive either.

A good witch hunt will at least point out a plausible witch.

jjoonathan3• 3 years ago

Even if the list were public, its enforcement will still involve a million subtle decisions that you have no hope of ever personally litigating with anything approaching fairness, but the moderators' jobs would be much harder because you can bet that every bad faith actor on that list would rush to clog up the forum with "informal appeals."

It always comes back to trust and experience.

r721• 3 years ago

You can switch "showdead" setting to "yes", and look for posts marked "[dead]" in https://news.ycombinator.com/newest (though those also can be from shadow-banned users, not based on domain only).

g42gregory6• 3 years ago

How are we better off with this list being secret and being kept away from us? Are you saying that we are better off not knowing what’s in this list? Why?

sanderjd3• 3 years ago

Because keeping this kind of list secret is an effective tactic in the battle against spam, and we are all better off with less spam.

belval• 3 years ago

Because it would cause largely pointless discussions about the content of the list. Maybe you feel like "yourpointofview.com" has really compelling content that you would like to discuss. The HN moderation team thinks it does not and ultimately it's their call.

dangerlibrary3• 3 years ago

Because domains are cheap, and hellbanning submissions works well to discourage spammers.

jfim• 3 years ago

It's the same way that shadow banning is effective, it lets people post links that are automatically flagged, saving moderator resources.

the-printer• 3 years ago

OP’s comment is better suited to be taken in reference to the amount of domains banned rather than the “secrecy” of the list. I’m interested in how users are worse off with regards to anything by not knowing what sites are banned.

idiotsecant3• 3 years ago

I think the 'free speech bros' characterization is this particular case might even be overly charitable, considering that OP appears to have created this account solely to make this post. I suspect they may lean more toward commercial interests and less toward ideological ones.

kordlessagain• 3 years ago

Things improve by banging on them periodically and seeing what falls off. There's a difference between unethical behavior and just pushing the limits of something with the hope it is improved. Then there's flat out trolling...

sigmoid10• 3 years ago

Depends on what you are looking for. Less moderated sites like 8chan drift heavily to the extreme political right and even into illegal stuff, but if that's your cup of tea you'll find a pretty active community there. It's just the liberal and scientific sites that need to weed out the crackpots, but that is an endless uphill battle.

hotdogscout2• 3 years ago

My concern is liberal and scientific communities have a history of looking the other way at Marxist violence. I wish I didn't need to choose which genocide deniers I'll have to cater to.

Pannoniae2• 3 years ago

From what I've noticed, free speech advocacy isn't really about "allowing everything" but more about not applying more censorship than what is necessary and not doing it arbitrarily. HN does quite well in this regard - there are principles in running the site which are adhered to and which are required to keep the community from deteriorating into social media-quality discourse.

For example, banning someone only for not liking their views is obviously legal and permitted almost everywhere, but a site/forum doing that doesn't adhere to the principles of free speech then. Which they don't necessarily have to do, but that's what free speech advocates are fighting for - for more places to adopt the conduct of not doing that.

joshstrange6• 3 years ago

From what I've noticed, free speech advocacy isn't really about "allowing everything" but more about not applying more censorship than what is necessary and not doing it arbitrarily.

That's a nice phrasing but it doesn't hold up to what I've seen at all. Everyone has a different definition of what is "necessary" and anyone that gets (shadow-)banned is going to scream "Censorship!". I've moderated communities and watched this happen enough times to know.

There simply is no objective way to handle it, it's always going to subjective and there will always be false positives. The alternative is a hellscape.

HN has rules it adheres to but if someone finds a loophole or continues to walk "right up to the line" they probably aren't going to be around for long. I tried moderating a community once completely to the "letter of the law" and it turned out horribly. People abused that and drove off people who were kind and had interesting things to contribute. What was left was a few people who stuck it out and a bunch of trolls who knew they could get away with it.

nvy2• 3 years ago

[...] not doing it arbitrarily. HN does quite well in this regard [...]

Hard disagree on the arbitrary part. What's considered "intellectual" or "curious conversation" depends entirely on dang's personal preference.

For example every time a Steve Jobs thread is posted, there inevitably are two major themes:

1. People fawning over how great a leader Steve allegedly was, despite his flaws

2. People going on about how Steve was a massive douchebag, in spite of his success with Apple

Neither of those topics are interesting, novel, or intellectually gratifying but only #2 is reliably censored by dang. CEO worship is on-topic here but criticism of those CEOs is apparently "not interesting".

Timon3• 3 years ago

From what I've noticed, free speech advocacy isn't really about "allowing everything" but more about not applying more censorship than what is necessary and not doing it arbitrarily.

That is what I kept reading from "free speech advocates", especially regarding Twitter etc. - those same people are now largely either silent or actively cheering on the same happening in regards to the other side right now. This is the fatal flaw with this definition: "not applying more censorship than what is necessary and not doing it arbitrarily" does not mean the same thing to all people, and it seems that the counter-mainstream doesn't give the mainstream more leeway. So is there some way this could be done without the pendulum just swinging fully the opposite way?

logicalmonster4• 3 years ago

This is a straw man argument. The issue isn't the potential presence of moderation, but the potential lack of transparency and the problems that could bring.

Good systems are designed to work by taking into account human weaknesses. The current HN system doesn't do this.

sshine• 3 years ago

What human weaknesses does the current HN system not take into account?

EA-31672• 3 years ago

Transparency sounds nice, but in practice doesn't that just mean, "Teaching the bad actors how to be better at what they do?" Maybe that works for a giant like Facebook, but how can it work on a site with very few dedicated mods?

What about the experience of using this site would be improved by that transparency? Compare those pros to the known cons and make a case.

very_good_man2• 3 years ago

X is now an attractive free-speech platform.

joshstrange6• 3 years ago

I needed a good laugh, thank you for that.

X is neither attractive nor is it a free-speech platform.

somethingreen2• 3 years ago

I do agree that in general case heavy moderation is preferable and most shitty communities are shitty because of too little moderation, not to much. But the rest of your reasoning gets destroyed by the fact that 4chan is, despite its age, infamy and continuous slide into lunacy, still an order of magnitude more popular website than HN and for a number of purposes, still the best community on the internet.

PawgerZ• 3 years ago

Can you explain what makes you see 4chan as "for a number of purposes, still the best community on the internet." The only purpose I can see being argued for is porn. For what other purposes is 4chan the best community? What makes 4chan the best community in these instances?

johnnyanmac14• 3 years ago

I'm not even sure if I can name 67,000 websites, so it's still a shocking number. Simultaneously, it does slightly worry me that the list isn't published should I end up submitting a "legitmate" link that is never seen.

archo3• 3 years ago

"About 67k” sites are banned

  For numerical context, I just looked up a list of current 'unique porn domains' which numbers 53,644.

Edit: Add; An additional 209,626 unique domains for adware, malware, tracking. etc.

_dujt• 3 years ago

All paywalled sites should be added to this.

Joel_Mckay• 3 years ago

That is really surprising... only 67k?

We usually ban 800k domains and 220k IPs on average, just to keep a small site tolerable. =)

unethical_ban• 3 years ago

The fact that many domains are banned should be more obvious. I would even say much of the list should be publicized, since I can't imagine it revealing sources and methods of blocking.

I agree with the stance that this is necessary though, to filter out spam and low effort/misinfo/blatantly scumbag sites from cluttering the front page.

bloopernova2• 3 years ago

EDIT: no longer Flagged by me.

Flagged because this is just going to devolve into arguing. I doubt there's going to be much insightful conversation happening in this thread.

HN works because of the good decisions made. I wish people would stop try to destroy that.

theultdev7• 3 years ago

I wish people would stop trying to destroy a conversation that others clearly want to have.

I don't see any arguing going on here, just a healthy debate about free speech balancing while still deploying spam blocking techniques.

This is a political and a technical issue. It's a fascinating problem with no perfect solution (yet?)

bloopernova2• 3 years ago

Understood. I've skimmed over the thread and I admit I was wrong. For what it's worth, thank you for convincing me to take a second look and remove the flag.

I do see that other(s) have flagged it though.

johnnyanmac14• 3 years ago

On the other hand, I feel a technical audience should understand the importance of peer review. I don't like the idea of stifling conversation because "trolls will ruin it". Yes, you can apply that as a blanket deflection of any sort of decision.

HN works because of the good decisions made.

from who's perspective?

_Algernon_3• 3 years ago

I don't mind HN being moderated by a BDFL. It's why I prefer this place compared to the alternatives.

dredmorbius4• 3 years ago

HN mods are staff, not BDFL, and at least one (sctb, a/k/a Scott Bell) rotated out. He's turned up occasionally as of late, as a mere mortal.

johnnyanmac14• 3 years ago

You don't mind until the B ends. Be it due to retiring, life changes, or simply revealing certain biases. Always stay on your toes.

_Algernon_3• 3 years ago

That's fair, but in that case I will find a new place with good moderation by a BDFL.

There exists no unmoderated (or barely moderated) community on the web of similar size to hacker news on which would want to spend significant time. There are no working alternatives, therefore the hypothetical possibility of a BDFL turning bad is irrelevant to my decision making.

jmuguy• 3 years ago

Thank you, randomly created brand new account, for reminding us all that moderation is the only thing keeping HN from being another cesspool.

bread_butter• 3 years ago

I’d imagine if Reddit were to do it we’d be up in our arms.

boomboomsubban• 3 years ago

Reddit does have a list of blocked domains, apparently they even wholesale block things on the .ru top level domain.

johnnyanmac14• 3 years ago

Reddit the business, or Reddit the subreddit managed by free mods?

The former does do this, and has for a very long time. The mods of each subreddit can ban whatever domain they want.

raverbashing• 3 years ago

Anyone moderately surprised by this is either a) naive or b) a spammer (or could be both as well) or a troll maybe

And of course never had to moderate/care for abuse for anything more than a personal blog (and I mean, even then)

Yes I'm sure HN is worse off for not allowing half-brained spam sites or some random dude's opinion piece with 3 readers that permanently live in a basement or something

johnnyanmac14• 3 years ago

consider me naive then. I'd rather ask and gain knowledge than pretend this is obvious and praise a lack of transparency.

pavlov2• 3 years ago

I genuinely wish Xitter were added to this list since they started playing games with login walls.

It’s annoying when an interesting headline comes up but the URL points to twitter.com, so it’s not worth clicking on to discover whether the site will let me read today.

There’s Nitter but it seems to be a whack-a-mole target that only sometimes works.

dredmorbius4• 3 years ago

I've suggested to dang in the past (via email) that HN consider spinning up its own Nitter instance for proxying Twitter submissions.

r2b2• 3 years ago

HN needs a @dang apprentice program so it can thrive for centuries.

afavour• 3 years ago

It's good to have a reminder now and then that some people have no clue how much work goes into effectively moderating a community to keep it functional.

Yeah, of course HN bans a ton of sites. I'm glad they do and I trust the mods to maintain that list correctly. If I feel they are banning sites unnecessarily I'll leave but so far I see no evidence of it.

yung_steezy2• 3 years ago

I believe archive.org is one of the sites because I tried to post an economist link using it to circumvent the paywall once and the post seemed to be shadow deleted

wmf2• 3 years ago

There's a rule to only post the original source. You can post an archive link in a comment.

kjs3• 3 years ago

Except there's a huge number of things where archive.org is the only source left.

yung_steezy2• 3 years ago

Makes sense in retrospect

ThrowAway1922A• 3 years ago

I've love to see the list for curiosities sake, but I imagine a solid 90% of it is either porn, or scam sites.

HN has good moderation unlike a lot of communities, Reddit included. The community has changed over the years, but it's still more tolerable than most and I'm happy with how it's moderated.

rexreed• 3 years ago

As Lesley Gore would say (paraphrased), "it's their party and we can cry if we want to".

TheAceOfHearts2• 3 years ago

I don't agree with flagging this post, it seems like an important discussion based on the large amount of activity.

As for my own views, I think it's fine that the list isn't public, it doesn't make much difference either way. I appreciate knowing that there's a list of banned sites.

My attempt at constructive feedback would be to have some place where a transparency report on moderation activity is shared. Not everything needs to be public nor immediately updated, I trust @dang's discretion on deciding what should be shared. But maybe it would be more ideal if there was a single place where you could find out that there is a moderated domain blacklist, rather than finding out through a random comment.

Overall I'm happy with HN's moderation and wouldn't want to impose too much of a burden on the team.

dang2• 3 years ago

It was correctly flagged because there was already a huge discussion about this in another thread, which this one is simply pointing to. I didn't even see the current submission until a few minutes ago, and I commented extensively in the other thread, answering every question I saw.

We don't want this sort of meta HN-eating-its-own-tail (I'll use that metaphor instead of the more vulgar one which came to mind first) in general, which is why submissions to news.ycombinator.com itself are banned on HN. This one shouldn't have been vouched for.

TheAceOfHearts2• 3 years ago

Oh, that's fair. I didn't realize that people had continued engaging on the original post.

hirundo• 3 years ago

Better would be a bottom-up approach where individual users can make block lists -- of sites and other users -- and freely share the lists with other users, who can adopt them with changes.

archo3• 3 years ago

A quick Reminder... Internet Websites in the USA are bound by Federal and State laws, regulations etc.

There is NO 'Absolute Free Speech'

"About 67k” sites are banned from submission on HN, the list is kept secret

Related Stories