Ask HN: How to de-dupe the HN front page

I've been using the HN client I wrote [0] for a couple of years now. It uses the official HN API. But I noticed that I get many duplicates every time I read it.

What's the best way to reduce duplicates? I always try to check recent posts before I post. Obviously if it has the same URL, it's a duplicate but it's not always that simple. Sometimes an article, like Elon Musk having Asperger's, is covered on many different sites and will appear 5 or 10 times.

(And yes, this is something that HN itself should try to prevent, but that's unlikely to happen anytime soon.)

[0] https://gabrielsroka.github.io/hn.html?small