Ask HN: Can HN prevent duplicates?

https://news.ycombinator.com/item?id=6053597

by trendspotter • 13 years ago

3 12 13 years ago

news.ycombinator.com

There has to be a solution for this kind of problem (duplicate posts like these two): check first https://news.ycombinator.com/item?id=6053531 and now check the other link https://news.ycombinator.com/item?id=6053424

tokenadult• 13 years ago

The duplicate link detector has a minimal ability to recognize duplicate links. The Hacker News community relies partly on the honor system to avoid excessive duplicate links. Some members point out duplicate submissions when they appear anyway. I'm not sure that this is a high enough priority for anyone on the HN programming team to devote a lot more effort to refining the duplicate submission detector.

trendspotter2• 13 years ago

It can't be that hard to let a machine find out that these two links are basically the same. I'm not talking about programming a artificial general intelligence (AGI) solution here:

(a) http://techcrunch.com/2013/07/16/estimote-creator-talks-abou...

(b) http://techcrunch.com/2013/07/16/estimote-creator-talks-abou...

chc• 13 years ago

Nobody's saying it's impossible. The point is that news.arc is more or less just a little project that Paul Graham works on in his free time, and the amount of features that could be implemented vastly outstrips the supply of Paul Graham's free time, so not every possible feature is going to be implemented.

pearjuice• 13 years ago

THIS THREAD HAS BEEN MARKED AS DUPLICATE

https://news.ycombinator.com/item?id=1012215

trendspotter2• 13 years ago

Thanks. I found a great suggestion to this problem over there:

"Maybe borrow a feature from Stack Overflow [or Get Satisfaction or Desk.com or Uservoice], which presents similar-sounding stories before posting? Some client-side JS which pulls the page, heuristically picks some "interesting" words, and runs them through search.yc?"

Here is the perfect and best of both worlds solution to prevent duplicates:

Step 1: (Algorithm) Automatically check the title (and maybe other metadata like keywords) of his submission against similar existing content on HN. Present these results to the user.

Step 2: (Human) Let the user's intelligence decide if his submission to HN looks like a duplicate of another user or not.

arh68• 13 years ago

And what, do away entirely with reviving old topics? I thought it was rather poignant to consider the old news article about Geocities' acquisition by Yahoo when Tumblr was recently acquired. Who knows what links submitted recently will still be interesting in several years? There is value in allowing duplicates.

ScottWhigham• 13 years ago

Just a word of warning for you - there is a threshold of flagging that, if you flag too many stories in too short of a time period, you will lose your flagging privileges permanently. It's an odd implementation, frankly, so be aware. It often rears its ugly head whenever there is a popular story here (most recently: NSA/Snowden) in which there are lots of dupes reported.

Ask HN: Can HN prevent duplicates?

Related Stories