Show HN: HN Trends - like Google Trends, but for Hacker News

http://hntrends.jerodsanto.net/

by sant0sk1 • 15 years ago

355 106 15 years ago

hntrends.jerodsanto.net

ysh4u• 15 years ago

This is really great!

yuvadam• 15 years ago

inb4 http://hntrends.jerodsanto.net/?q=bitcoin

bobbles• 15 years ago

similar results.. http://hntrends.jerodsanto.net/?q=airbnb

ntoshev• 15 years ago

Awesome work!

Can you normalize it against total text volume? Number of mentions always goes up with time as HN grows, % of posts mentioning will be a better measure of mindshare.

sant0sk114• 15 years ago

Thanks so much! Hmm, good idea on % of posts, though most of the value I believe is in comparing multiple terms, in which case the raw number of mentions isn't as important as the relation of the trend lines.

akavi• 15 years ago

That's not entirely true.

For example, the first thing I searched for was "Haskell", curious to see if it had gained or lost popularity in the past couple of years. The graph has an upward trend, but without knowing what the trend for total volume on HN is, that information is useless.

edge17• 15 years ago

first of all, very cool.

how is it normalized? every single thing I type in has an upward trend, which to me just suggests that interest in Hacker News has increased over the last few years... and I don't really need a chart to tell me that since I live in silicon valley :)

sant0sk114• 15 years ago

Good point. I wasn't really thinking about normalization as I was more interested in comparing trends on multiple terms, but I don't think it'd be too much work to change the algorithm to account for HN's growth.

suking• 15 years ago

Yah, you could easily do it by total # of posts to account for the growth. Cool stuff man.

hammock2• 15 years ago

The trends on multiple terms are actually easier to read after normalization anyway. Right now the power-law curve growth of HN in total is making it hard to read the difference in growth between, say, FB and twitter. Easier for our brains to process if you took the ln() of them - which is definitely the fastest way to fix the charts - or normalized in some other way.

retube4• 15 years ago

Came to say exactly this. The chart seems to start (by default anyway) at the beginning of 2007. I'm guessing the user-base and thus activity of HN has increased considerably since then. So yes it would be good to be able to re-normalise in some way. By the number of registered users would probably be the simplest. By number of comments posted would be another, by number of submissions or even number of words would all be interesting.

Btw - my suggestions got me thinking - I assumed this is looking at posts and comments. Whether just submission titles or if includes comments will affect best way to normalise.

sant0sk114• 15 years ago

It uses submission titles AND comments. How would you suggest going about normalization?

hammock2• 15 years ago

Doesn't seem to be normalized at all. The easiest way to fix would be to chart the y-axis on a log scale.

sant0sk114• 15 years ago

Just an update to let you know that the site now accounts for HN's growth. I used retube's suggested adjustment scheme. Let me know what you think!

arturadib• 15 years ago

Is it possible to somehow normalize the numbers by the total number of posts/comments, so that trends across the years do not mostly reflect the user base/activity growth at HN?

Awesome work thus far though.

sant0sk114• 15 years ago

Yup, this seems like an obvious improvement as I believe you're the 3rd to mention it already :)

I'll definitely do that next, since it's not too hard to implement and seems like a big win when you are just trending a single term.

patrickaljord• 15 years ago

http://hntrends.jerodsanto.net/?q=android%2C+iphone

tintin• 15 years ago

http://hntrends.jerodsanto.net/?q=android%2C+iOS

jr62• 15 years ago

http://hntrends.jerodsanto.net/?q=google%2C+apple

solipsist5• 15 years ago

These trends are fairly interesting...

http://hntrends.jerodsanto.net/?q=pg

http://hntrends.jerodsanto.net/?q=y+combinator

http://hntrends.jerodsanto.net/?q=pg%2C+tptacek%2C+patio11%2...

http://hntrends.jerodsanto.net/?q=2007%2C+2008%2C+2009%2C+20...

http://hntrends.jerodsanto.net/?q=android%2C+iphone vs. http://www.google.com/trends?q=android%2C+iphone&ctab=0&...

http://hntrends.jerodsanto.net/?q=show+hn%2C+ask+hn%2C+tell+...

http://hntrends.jerodsanto.net/?q=mac%2C+gruber%2C+steve+job...

Don't forget thought that these trends include the user's own comments and submissions in addition to mentions of them in other users' comments.

suking, this is truly awesome. It immediately joined my bookmark folder of favorite HN tools. I'd be curious, though, if whether or not it would be possible to filter out user names when counting how many times a word has appeared on the site. Or would that be too much of a challenge?

sant0sk114• 15 years ago

I was surprised to learn that the search API returns hits on usernames when they appear outside of the post/comment content.

I'm looking now to see if there's an easy way to filter around it. I've also been emailing with one of the ThriftDB guys and I'll ask him as well.

If not, it'd be very difficult to filter those out because I'm not currently (nor would I want to) retrieving all the matching items, just the number of hits.

solipsist5• 15 years ago

Oh yes, I forgot you were using the HNSearch API. By the way, I hope you win the API Contest (don't they decide tomorrow?). You for sure get my vote!

qF• 15 years ago

This is actually pretty easy to do! You can apply weights to fields, so just apply 0.0 to the username field - or that's how I did it anyway.

Your entry looks very nice, cool to see others had a similar idea. :)

pitdesi• 15 years ago

What else is in that bookmark folder of your favorite HN tools?

solipsist5• 15 years ago

Here are some of the various HN tools and interfaces I have collected over time:

hckr news - http://hckrnews.com/

Sortable HN - http://hnsort.com/

hnvue - http://www.hnvue.com/

HackemUp - http://www.mrspeaker.net/2011/04/11/hackernews-bookmarklet/

Search Y Combinator - http://searchyc.com/

WWHNS (What Would Hacker News Say) - https://github.com/marcua/wwhns/

HNSearch - http://www.hnsearch.com/

Visual Search - http://hnsearch.heroku.com/#h

Hacker News Resources - http://hnresources.com/

Kalimba - http://hn.embed.ly/

The Best of ... Hacker News - http://bitovod.com/hn/best-of

HackRSS - http://www.hackrss.com/

HN Trends - http://hntrends.jerodsanto.net/ ;)

levesque• 15 years ago

Hacker news daily - http://www.daemonology.net/hn-daily/

trb• 15 years ago

Some rather social search terms:

http://hntrends.jerodsanto.net/?q=they%2Cwe%2Cyou%2Ci

I don't understand why "they" doesn't occur. Maybe a bug?

http://hntrends.jerodsanto.net/?q=they+deserve%2Cwe+deserve%... http://hntrends.jerodsanto.net/?q=they+should%2Cwe+should%2C... http://hntrends.jerodsanto.net/?q=they+failed%2Cwe+failed%2C... http://hntrends.jerodsanto.net/?q=i+am%2Ci+was%2Ci+will+be

JohnnyBrown• 15 years ago

Idle speculation: Wouldn't it be cool if we plotted a user's total karma next to the trend of their name and it turned out to look like an integral?

iskander• 15 years ago

For some of us, it will look more like a martingale :-).

shellehs• 15 years ago

Can not click the point to view all the relative articles, but, it's sort like painting and counting looks so cute, ^_^

pvilchez• 15 years ago

I searched for my own username expecting a flatline, but there were a few hits in the last few years. Flattering, but I'm fairly certain that hackers have not been talking about me. ;)

Perhaps you should only query titles and text, not usernames?

edit: for example, searching for pg would include his own submissions as well as mentions and Ask PG subs.

sant0sk114• 15 years ago

Strange, I didn't expect the search API to return results for a name that's in the by-line of a comment, but it sure does:

http://www.hnsearch.com/search#request/comments&q=pvilch...

This is something I should definitely work around in the future. Thanks for bring it to my attention!

andres• 15 years ago

You can exclude authors by setting the weight to 0 in your API call:

&weights[username]=0.0

digitalnalogika2• 15 years ago

Nice work! I had the same exact idea for the contest and bought the hntrend.com domain 2 weeks ago, too bad I was too lazy to finish it :)

franze2• 15 years ago

well, looks like there is a service which would deserver that domain...

digitalnalogika2• 15 years ago

http://twitter.com/#!/sant0sk1/status/82564569708503040

vnorby2• 15 years ago

Had the same idea myself as well, bought hnstats.com (although I admit, I looked for hntrend and hntrends.com)...not enough time in the world to do all of these things. Great job with this.

staunch• 15 years ago

On an IPad1 the charts render so slowly it's watching a race between brightly colored snakes. Very suspenseful.

Please do normalize the results.

driverdan• 15 years ago

The charts are SVG and iOS's implementation of SVG is stupid slow.

kristofferR• 15 years ago

It works great on the iPad 2' with animations and everything

iamwil• 15 years ago

http://hntrends.jerodsanto.net/?q=fuck%2C+shit%2C+cunt%2C+co...

I put in 5 of the 7 George Carlin dirty words. Looks like 'motherfucker' has a recent surge.

http://hntrends.jerodsanto.net/?q=piss%2C+tits%2C+cunt%2C+co...

If you want to see it without the top two words dwarfing the other five.

corin_• 15 years ago

Possibly from "The Motherfucking Manifesto For Programming, Motherfuckers"?

false• 15 years ago

I asked it about ios and android and it broke :( http://hntrends.jerodsanto.net/?q=ios%2C+android

sant0sk114• 15 years ago

Should be back now. I had to restart the heroku process. Sadly, no indication why it crashed :/

foenix2• 15 years ago

You may find this to be a more useful comparison: http://hntrends.jerodsanto.net/?q=iphone%2C+android

cheez• 15 years ago

It's cool but doesn't always seem to chart the trend. I have to force a refresh. Using Chrome stable.

elsewhen• 15 years ago

we may be in a bubble about talking about a bubble: http://hntrends.jerodsanto.net/?q=bubble

yawniek• 15 years ago

http://hntrends.jerodsanto.net/?q=bubble%2C+fuck interesting...

a very nice tool!

nxn• 15 years ago

Hah, I think that's a neat way to visually normalize against the traffic increase trend.

vijaydev• 15 years ago

http://hntrends.jerodsanto.net/?q=ruby%2Cpython

http://hntrends.jerodsanto.net/?q=rails%2Cdjango

Interesting that Ruby stays below Python but Rails stays well above Django.

grok1fy• 15 years ago

There are similar metrics for job openings. It seems plausible that Python is becoming an all-purpose language while the focus for Rails / Ruby is on web apps. If true, this would be reminiscent of Perl v. PHP.

franze2• 15 years ago

XML vs Json http://hntrends.jerodsanto.net/?q=xml%2C+json that's close

Jarred• 15 years ago

These trends don't account for traffic increasing to the site (and therefore posts/comments) as time progresses. That makes this less useful.

Steko• 15 years ago

Agreed would love to see a normalization option that transforms it into a % of submissions in each period.

georgieporgie• 15 years ago

Very cool.

Amazingly, C# is talked about precisely as often as C++!

http://hntrends.jerodsanto.net/?q=c%2B%2B%2C+c%23

;-)

solipsist5• 15 years ago

I think it may ignore the #, ++, along with various other symbols. See this link (you may have to refresh once or twice if it doesn't work the first time): http://hntrends.jerodsanto.net/?q=c%2B%2B%2C+c%23%2C+c-%2C+c...

csomar• 15 years ago

For the people who says that HN discuss less about startups, the trends actually shows something else: http://hntrends.jerodsanto.net/?q=startups

rmk• 15 years ago

Pretty cool visualisations. Just out of curiosity, why did you choose HighCharts? Did you look at jquery FLOT and choose HighCharts over FLOT? Or were you unaware of FLOT altogether?

http://code.google.com/p/flot

sant0sk114• 15 years ago

I chose HighCharts because I have experience with it and knew right away that it'd get me where I wanted to go.

I had never heard of FLOT. Thanks for introducing me to it, it looks cool and I'll check it out for future projects!

swah• 15 years ago

Is it normalized against the growth of the website?

sek• 15 years ago

http://hntrends.jerodsanto.net/?q=scala%2C+erlang%2C+clojure...

interesting indeed

jhuni• 15 years ago

It would seem that NodeJS is taking away some of the share of functional programming languages.

mikemoka• 15 years ago

this is awesome, it would be a dream if advanced queries could be comparable though

teddytruong7• 15 years ago

this is great, awesome how its so fast.

dclowd9901• 15 years ago

What the hell happened with Javascript in mid 2009?

http://hntrends.jerodsanto.net/?q=coffeescript%2C+jquery%2C+...

cpeterso• 15 years ago

Node.js was released in mid-2009. Did it launch a surge in JavaScript popularity or just ride it?

dhbanes• 15 years ago

Let us never forget the first great hacker news bubble.

http://hntrends.jerodsanto.net/?q=nickb

kuahyeow• 15 years ago

I like the animation, but it will be an improvement if the chart max stays and not jump up and down

sant0sk114• 15 years ago

The problem with that is that I have no idea what range the terms queried will be in beforehand. Any ideas how to accomplish this?

rsoto• 15 years ago

You could animate after you've received all the data, then set the Y axis into the max of the values. Anyway, good job! Thanks for sharing.

aymeric• 15 years ago

Out of curiosity: http://hntrends.jerodsanto.net/?q=ycombinator%2C+techstars

nagnatron• 15 years ago

vim/emacs http://hntrends.jerodsanto.net/?q=vim%2C+emacs

tomp• 15 years ago

http://hntrends.jerodsanto.net/?q=bitcoin

Looks like a bubble...

matt1• 15 years ago

Hey Jerod. I own the domain HNTrends.com, which I used for a similar project about two years back [1][2]. If you plan on keeping this project up, I'd be happy to transfer the domain name over to you. Shoot me an email: matt@leandesigns.com.

[1] http://news.ycombinator.com/item?id=810112

[2] http://www.mattmazur.com/category/hntrends/

noduerme• 15 years ago

Lol. According to HN, Chrome should be the world's top browser. http://hntrends.jerodsanto.net/?q=safari%2C+firefox%2C+inter... -or- http://hntrends.jerodsanto.net/?q=safari%2C+firefox%2C+IE%2C... (with "IE")

Great site by the way. Nice job.

foenix2• 15 years ago

An observation about geographies: http://goo.gl/4BUFL

jamesbritt• 15 years ago

Very, very slick. And now I can can graph my increasing time spent on HN :)

http://hntrends.jerodsanto.net/?q=jamesbritt

kristopher• 15 years ago

I found this interesting: "Hacker News" vs. "HN"[1]

As the site gains reputation/users you can see that many switch to the abbreviated version.

[1] http://hntrends.jerodsanto.net/?q=hacker+news%2C+HN

hmart• 15 years ago

http://hntrends.jerodsanto.net/?q=bitcoin%2C+groupon%2C+bubb...

OwlHuntr• 15 years ago

Obvious causation: http://hntrends.jerodsanto.net/?q=dolphins%2C+PROTECT+IP

rwebb• 15 years ago

slick!

johnx123• 15 years ago

From where HNSearch is pulling data? Do we have any dump for HN?

pavpanchekha• 15 years ago

Is there any way to normalize by number of users? (Or am I missing something?)

Concours• 15 years ago

Technews trends, what are we reading and sharing here: Techcrunch vs Mashable vs Readwriteweb vs CNN vs Techmeme

http://hntrends.jerodsanto.net/?q=techcrunch%2C+mashable%2C+...

sdfjkl• 15 years ago

Excellent work! http://hntrends.jerodsanto.net/?q=excellent

robinwarren• 15 years ago

Hi, I created a similar tool using the search API, but with lniks to the top stories (by points) over time

http://touchwoodsoftware.com/yctrends.html

Feel free to take that idea if you like, I think it's interesting not just to see the shape of the trend but also the stories which created the peaks.

wvanwazer• 15 years ago

Awesome tool, but just a heads up: I tried searching for "C" and "C++", and it gave me the exact same line each time.

tilt• 15 years ago

(Cross-post) There could be a spike in HN Trend's traffic http://twitter.com/#!/timoreilly/status/82824209721597952

Show HN: HN Trends - like Google Trends, but for Hacker News

Related Stories