Clearing up some things about LinkedIn mobile’s move from Rails to node.js

http://ikaisays.com/2012/10/04/clearing-up-some-things-about-linkedin-mobiles-move-from-rails-to-node-js/

by ikailan • 14 years ago

205 32 14 years ago

ikaisays.com

rhizome• 14 years ago

And the blog-to-commentary-to-blog cycle begins anew.

uncoder0• 14 years ago

and then come the posts about how they scaled their custom blog backend to cope with the hn front page traffic.

ikailan3• 14 years ago

(I'm hosted on Wordpress; that's their problem now)

klochner• 14 years ago

This sounds much more sane than "node is 20x faster than rails", thanks for validating the assumption most of us were making.

davidlumley• 14 years ago

This. I wish there had been more context in the original article.

JimmaDaRustla3• 14 years ago

I interpreted the original article that their new node.js approach was wildly different than a server-based MVC framework.

Sure, they ended up with 20x greater speed, but never did they say that it was because node.js. They even provide a high-level detailed description of all the things they did with their new approach to achieve this performance.

marknutter• 14 years ago

Yes, but the title on HN implied that. We need to be better about writing our submission titles, but hey, sex sells.

mhartl2• 14 years ago

I met Ikai through the Silicon Valley Rails Meetup, which I co-hosted back in 2008-2009 and which met at LinkedIn HQ in Mountain View. This post is a great contribution to the recent discussion about Rails at LinkedIn, and I hope it gets the attention it deserves.

ikailan3• 14 years ago

Hey Michael!

Was there a recent discussion? I'm fairly certain everyone has moved on from Rails, and I'm not sure if they're still using it anywhere at LinkedIn. There are a few folks using JRuby, but I believe they're using Sinatra:

https://www.google.com/search?q=jruby+sinatra+linkedin

mhartl2• 14 years ago

I haven't been back in a while, so I've got no inside information. But it's great to hear from you!

eta_carinae• 14 years ago

> And those requirements kept growing. If my calculations are correct, the standard setup for engineers now is a machine with 20 or more gigabytes of RAM just to RUN the software.

Close. In 2011, all the engineer desktops got upgraded to 36Gigs. At the time, the eng department still hadn't figured out how to deploy without duplicating hundreds of jar files everywere.

ikailan3• 14 years ago

Hah, I bet all the new engineers were like, SWEET! I get a system with 36 gigs of RAM!

Then their hearts sank when they realized why.

davedx2• 14 years ago

Wow, that sounds like fun. What exactly is in the LinkedIn stack that requires so much memory?

ucho• 14 years ago

Nowadays I get OOM while running only FireFox and Netbeans - and I am not even running software I develop on my box :P

brown9-2• 14 years ago

Speculation, but it sounds like the main webapp you see at linkedin.com is a single monolithic application.

joshhart• 14 years ago

Systems actually come standard with 48 gigs :) As it turns out, that's really useful for the part of my job requiring data modeling.

Somebody suggested LinkedIn is one large application, which is absolutely false. It's composed of many small applications, each deployed as a WAR file since the non-Node.js part of the site is on the JVM. There's a lot of overhead that comes from deploying this way. It doesn't matter in production or staging, but it adds up on a desktop.

At this point in time though, dev machines have stopped growing because we have shared stacks you can deploy a couple pieces of the application against if you want to test the whole thing.

davedx2• 14 years ago

This is a great post for validating management concerns about pulling in sexy new technologies for the hell of it. Every place I've worked I've been unable to convince management to use e.g. Rails (5 years ago) or node.js (recently). Even though I love these technologies and wish I'd had more time in full-time employment to learn and play with them, I understand and appreciate the risks implicit with adopting a shiny new technology in your company's IT dev/production environments.

It's also a great post illuminating how in hindsight some things can be really obvious (that building a high capacity web service dependent on a single-threaded server will give you problems down the road), but at the time it's not always easy seeing the woods for the trees.

For me though, the big takeaway was that one line summary: "You’re comparing a lower level server to a full stack web framework." Node.js has a pretty nice library/module ecosystem now, but for a complete full-stack solution with maximum productivity I would venture that there is nothing out there that compares to Rails currently.

cmccabe2• 14 years ago

This is a great post for validating management concerns about pulling in sexy new technologies for the hell of it.

At the end of the day, the Rails stuff got the job done. LinkedIn stayed up and was able to grow and add mobile features during that time. The current solution, the node.js stack, is even newer than Rails. So no, I don't think this validates management desires to stay with old technology.

Um, the new solution is single-threaded too. Threading and concurrency are not the same things.

dasil003• 14 years ago

> This is a great post for validating management concerns about pulling in sexy new technologies for the hell of it.

But their old technology is requiring towers with 36GB of RAM. It seems like more of a choice between the devil you know vs the devil you don't. Obviously there needs to be a cost-benefit analysis even if it's imperfect. Otherwise you'll get stuck in a local maxima and before you know it you're paying $10 per million cycles of a mainframe running batch COBOL jobs maintained by old men who are dying faster than you can replace them.

trung_pham• 14 years ago

Node.js is old news. Time to move on to GoLang. :)

cmccabe2• 14 years ago

Revel is the way to "go"...

jeremyjh• 14 years ago

Please do not even joke about this.

b3tta• 14 years ago

It's definitely true, that node.js isn't a full fledged framework, but I still wrote several projects using it and you know what? I don't regret it, as much as I don't regret my move from C++ to C over the past years. And for the memory usage: Yes, even my biggest project never needed more than 100MB of RAM (when not using the cluster module).

But I completely agree with "the rewrite thing". I guess the other factors made it necessary to still do it....

lolwutreddit• 14 years ago

Wow, linkedin sounds like a place I really want to work!

Billy Madison: "High school is great. I mean I'm learning a lot. And all the kids are treating me very nice. It's great."

JimmaDaRustla3• 14 years ago

The concept that I got from the main article was that they tailored their application for "long-lived connection" to avoid multiple resource calls to make their web server more responsive. They also mention things like using "aggressive cacheing", "storing templates locally", "using timestamps to stream only required resources", "rearchitecture and rewrite."

Never once did I ever feel this article was advising that node.js was superior to RoR - they only every justified, at a high level, a way better approach (in terms of server load) to an "MVC" like architecture by leveraging client side frameworks and technique to lessen server load.

The author of this article also makes it clear at the end by stating that comparing the solutions is apples to oranges, but so did the original article...so I don't get the need for "clarification".

EDIT: I retract my "way better" statement - I mean "way better" in the sense of server load.

nirkage• 14 years ago

Mongrel isn't and wasn't single-threaded. Rails was.

xradionut• 14 years ago

"•Firefighting? That was probably a combination of several things: the fact that we were running MRI and leaked memory, or the fact that the ops team was 30% of a single guy."

:o That may explain why they had spam and security issues.

tedunangst• 14 years ago

He's talking about a mobile site that was using a public API. They had no more access to the password database than you or I do.

Clearing up some things about LinkedIn mobile’s move from Rails to node.js

Related Stories