Show HN: Blogosphere – Discover independent technical blogs

by BillFranklin • 5 years ago

200 48 5 years ago

Hi, I think discovering blogs and articles by other engineers is quite hard. Particularly non-commercial content.

I made blogosphere last Summer in order to improve discovery. It’s seeded with personal blogs that have at some point appeared on the HN front-page.

The intent of the site is to recommend blogs and articles to you, with the objective that it’ll get better the more you use it.

The site uses a co-occurrence matrix to recommend articles and bloggers to you based on the articles you ‘star’ and topics you follow. If you’re not logged in, the site will show you popular articles. It doesn’t track what you read, and doesn’t have google analytics. I have some ideas for improving recommendations (i.e. people like you also like.., and improving the auto tagging of articles), but I thought I’d get feedback on whether people want this first.

The site is at an MVP stage, there will be bugs and missing features, but feedback on whether this is interesting to you would be useful.

m_a_g• 5 years ago

I wonder, do you handpick every blog post or did you write some sort of a crawler?

BillFranklin9• 5 years ago

I started with the contents of my own RSS feed, and then crawled Hacker News for popular blogs, and then crawled the feeds of those blogs (in order to get posts that didn't get upvoted on HN).

The time consuming part was manually auditing blogs for quality (and to ensure it's a personal blog, not e.g. a BBC feed). I came up with some heuristics (i.e. blogs with github.io subdomains that got upvoted were often good so I blanket approved those) to speed things along, but it did take a while.

jarbus• 5 years ago

Nice work, I really like this idea!

godelski• 5 years ago

It doesn’t track what you read, and doesn’t have google analytics.

Just wanted to say thanks for minimizing tracking of users.

soldehierro• 5 years ago

Wow, this is brilliant. I know my way around a computer and have programmed before, well enough to understand the discussions around here, but I am decidely not a "tech person" by any means. I think this concept would be really great if expanded to fields outside tech.

polote• 5 years ago

https://findka.com/ is a similar service

it’s seeded with personal blogs that have at some point appeared on the HN front-page.

also this https://hnblogs.substack.com/ newsletter (than I manage) send you everyday with non commercial blogpost of HN of the day, if that can help seeds your aglo

m-i-l• 5 years ago

Looks good. I wonder if there'd be any interest in combining with my side project, which is a search engine for user-submitted personal and independent websites (and also search as a service). I could do with a nicer user interface and more structured data like yours, and perhaps you could benefit from features like user-submitted sites and scheduled (plus on-demand for verified sites) reindexing. Details in my profile, with links to the (open source) code on the site.

lastlegion• 5 years ago

How do you add a blog?

BillFranklin9• 5 years ago

Sorry, I haven't added a way for users to add blogs yet. If you could send me a link to an RSS/Atom feed I'd be happy to add more blogs. Auditing blogs for quality is quite time consuming so it'd be a good feature to add. Feel free to DM me (link in bio) if you'd prefer to send me a link privately.

lugoues• 5 years ago

Love it, great work!

One recommendation I have is Jacob Kaplan-Moss' blog: https://jacobian.org/index.xml

qznc• 5 years ago

More sources:

https://hnblogs.substack.com/

https://www.dannysalzman.com/2020/04/08/analyzing-hn-readers...

elliotbnvl• 5 years ago

Refreshingly MVP broken design. Love it (non sarcastic, shows a focus on the value add).

But I can’t create an account on mobile (iPhone Pro Max). :(

BillFranklin9• 5 years ago

Thanks for taking a look, sorry the login on your mobile isn't working, I've added that to my bug tracker.

My focus so far has been making sure recommendations are good enough, but it would be nice if one could use the site on a smartphone.

searchableguy• 5 years ago

Good stuff here. I have tried to build something similar before but nothing came out of it.

1. Repeated content.

I don't want to read what I have already read from a different place. I built a simple LSH based filter. I experimented with few ways to sort out text and process it. It worked.

2. Filter controversy.

I tried Bayesian fitler initially and moved to logistics regression using tf-idf. I settled on Bayesian because my dataset became very expansive. I used news-site corpus and manual entries from reddit/HN. I used sentiment analysis using a dictionary but it worked only in very specific cases. I do like some controversial and pessimistic content.

3. Filter clickbaits.

I couldn't filter the clickbait and gave up. There are ton of clickbaits on HN which I loved after I read them but ton of them are terrible and a huge waste of time. No reliable way to distinguish based on an article too. Length is not a good feature, negativity is not a good feature (I like to read strong opinions from say, founder of an open source analytics company criticising a big company for malpractices and how they fix those), sentence complexity is not good feature, and ton more.

4. Relying on user input is bad.

I read ton of nonsense everyday that I could go without knowing. I click on those links and that is a not me saying you should show me more of that. I don't want to do manual work of training something either. It's friction and I don't like it.

anyway, good luck on your site!

BillFranklin9• 5 years ago

Thanks for your feedback. I appreciate the tip on the LSH filter. One thing that has helped so far is restricting the content manually and with heuristics, as it prevents clickbait/controversial/malicious content. If I can implement collaborative filtering in future, I'll need to think about how to weight 'stars' from users to prevent malicious behaviour and improve recommendations, but I am getting ahead of myself :)

marcodiego• 5 years ago

Someone please reboot the webring idea, include a search feature and kill google!

Something I'd love: a search engine for personal websites.

pabs3• 5 years ago

Drew DeVault's blog uses openring to add a webring to his articles:

https://git.sr.ht/~sircmpwn/openring https://drewdevault.com/

mitchdoogle2• 5 years ago

Millionshort.com allows you to leave out the top x sites (100/1000/100000/1000000) from a search engine. Often it will give me some personal blogs in the results. Also nice for finding random information about some topics

8bitsrule• 5 years ago

Great tool, Thanks! After 15 minutes trying a few obscure searches, great -not- seeing them dominated by familiar sources. (Deeper research does -not- need endless repeats of introductory stuff!)

BillFranklin9• 5 years ago

I had a crack at that with the blog directory (https://bilbof.com/blogosphere/blogs), but it's pretty limited!

One thing I thought would be cool would be an API for similar blogs or articles, pretty much opening up an internal part of blogosphere. I think one could use that to provide a webring.

mkr-hn• 5 years ago

It's like a niche Technorati.

e3bc54b26• 5 years ago

Some time ago this discussion happened, someone suggested a search engine. It was not much of a search engine but random site finder, but the database only comprised of user submitted sites that were expressly approved.

I found random posts from people's garages and amazing projects, one of them being a dad's nanosecond counting clocks to prove time dilation to his kids over the weekend.

I was pretty sure I bookmarked it, but I don't find it anywhere now.

mitchdoogle2• 5 years ago

Random site finder sounds like stumbleupon, which was my go to source for wasting time 15 years ago. You could visit random pages for hours on end and every single one would be fascinating. You could set up an account and choose categories if you wanted or you could just do it without an account and see what it gave you.

Web rings are another thing that comes to mind. They were super popular in the 90s and often very niche. A site joined a ring and placed a widget on their page and you could click it to jump from site to random site in the same ring.

I think about these things and wonder why there seems to be fewer tools these days for that kind of random discovery. Then again, maybe I'm just nostalgic for a time in my life in which I could spend more time wasting time at a computer.

e3bc54b26• 5 years ago

There are tools for discovery and they are better than ever. Think wilby.me and awesome-* lists.

But there is so much noise, so very very much noise and utter crap out there that it is terribly hard to find what we are looking for.

I sympathise with you, I used to be proud in my Google-fu, saying if I came across something on the internet, I can find it again, but that's just not true anymore. It's not finding needle in a haystack anymore, now its finding needle in Mount Craperest.

fctorial• 5 years ago

blogsurf.io ?

e3bc54b26• 5 years ago

No, this wasn't it. My google-fu is also failing me.

e3bc54b26• 5 years ago

Found it in discussion linked in sibling comment:

Search engine : https://wiby.me/

Time dilation post : http://www.leapsecond.com/great2005/tour/

lijf• 5 years ago

https://gossipsweb.net/ ?

https://news.ycombinator.com/item?id=26446516

e3bc54b26• 5 years ago

Found it in discussion linked in sibling comment:

Search engine : https://wiby.me/

Time dilation post : http://www.leapsecond.com/great2005/tour/

leonry2• 5 years ago

There was recently this discussion: https://news.ycombinator.com/item?id=26506126 (in particular https://news.ycombinator.com/item?id=26507503 and comments)

Another recent one that came up is https://news.ycombinator.com/item?id=26618000

Is the one you look for listed in one of the threads?

e3bc54b26• 5 years ago

Thank you! Found it within first link you shared!

It was wilby.me

This was the time dilation post : http://www.leapsecond.com/great2005/tour/

segalord• 5 years ago

Thank you for this, I'll proceed to add them to my RSS feed

pknerd2• 5 years ago

It does not contain my blog :P

BillFranklin9• 5 years ago

Hi Adnan! Sorry for the delay, I've added your blog: https://bilbof.com/blogosphere/blogs/33759.

pknerd2• 5 years ago

Thanks, Sir! :)

teitoklien• 5 years ago

I just started using it to find articles , and after reading a few , I already love it !

Thanks a lot for making this.

pentlander• 5 years ago

Some quick feedback on the mobile layout, there isn't a lot of the actual blog post text visible due to the margins. I would suggest allowing the url and text to span the full width of the page. Also I would increase the margin between the title and the icon and star button, and potentially remove the word "star" to give more breathing room. It might also be nice to make the sidebar buttons an overlay to give more width as well.

fragileone• 5 years ago

I'd like to see a RSS page for the Home and Popular feeds since I've moved all my news sources to my RSS reader.

f0rfun• 5 years ago

I tried it. Already benefitted. Thanks for putting this together. Let us know how we can contribute, if any, to the features/code.

Thanks again.

bobbydreamer• 5 years ago

I have my favorites here https://bobbydreamer.com/irevere

gHeadphone• 5 years ago

I love your site Bill. It’s a great idea, I use HN as a source, but the submission rate is so high that I often miss great posts. Could I suggest my blog as a possible addition please? Markgreville.ie

Show HN: Blogosphere – Discover independent technical blogs

Related Stories