Pushing the performance limits of Node.js

http://www.jut.io/blog/2015/pushing-node-js-performance-limits

by henridf • 11 years ago

31 21 11 years ago

fein• 11 years ago

one effective way that we get high performance out of node.js is to avoid doing computation in node.js.

And therein lies the problem with node. Its great for rapid prototyping, but things fall apart pretty quickly with CPU laden tasks. That and the whole single threaded thing with node (technically JS) can be a wrecking ball when one task takes too long.

NathanKP4• 11 years ago

I've done CPU laden tasks many times in Node. The key is how you break up the heavy task into pieces that can be distributed more evenly on the event loop instead of blocking it. For example processing a huge data stream is typically done by breaking it into chunks where each chunk can be processed in a few ms, rather than doing one giant blocking operation on the entire stream.

Done properly this also leads to a low memory process that will grab data from an incoming socket stream, transform it, then push that data back out through an outgoing socket stream, only ever keeping a few MB of the stream in working memory at a time. Slap a cluster of these streaming processes together running on a single box to utilize all CPU cores and you can easily push many Mbps of IO through your server quickly and efficiently.

Additionally even if you do have a heavy blocking system IO operation that you can't break up very well you can increase `UV_THREADPOOL_SIZE` when needed to reduce the impact of the blocking.

SignMeTheHELLUp• 11 years ago

But now you've just implemented thread context switching by pulling it's implementation details into your algorithm... Just use a language that supports threading if you have real work to do...

demmer• 11 years ago

(Disclosure: I’m the VP Eng at Jut)

As Dave pointed out in the blog, we’ve definitely run into our fair share of challenges with the inherent limits of node.js. But on the whole, we’ve found it to strike a good balance between speed of development and performance. In addition to some of the techniques talked about in the post, we’ve also dropped down to C++ add-ons where we have really critical performance sections, and we do coarse-grained load balancing between multiple node processes to distribute computation on multiple cores.

We also have a good bit of shared code between the browser and server (including the Juttle language compiler and runtime) that has been hugely beneficial for us.

YMMV of course.

kaoD2• 11 years ago

Same problem with hammers. They're great for nailing, but nearly useless when eating soup.

What I'm trying to say is: use the right tool for the job. If your code is CPU intensive either don't use Node or use it exclusively for the IO-bound tasks and spawn jobs (or use a message queue, or whatever) for your CPU-bound needs.

richmarr• 11 years ago

There are two conflated points here.

... things fall apart ... a wrecking ball...

This one is FUD.

I doubt there are many left on HN or anywhere else that hasn't heard this argument, and a similar proportion who don't know the solution (which is that you use clustering or worker processes for tasks the block the process more than your use case can absorb).

Its great for rapid prototyping...

This one is true, it is great for rapid prototyping, but the implied point that it's not good for production apps is demonstrably false (as proven by the number of serious and respected companies happily using it in production).

There's an argument to be made about node.js being a form of technical debt, and that's a reasonable point, but it also needs to be made in the context of the huge value of that technical debt.

Update: I actually think Node.js is a contributor of technical debt but that it's worth the cost for the use cases I usually deal with (see point 7 on my post: https://medium.com/@richmarr/technical-debt-and-why-most-peo...)

xlm1717• 11 years ago

I had never read about that 1.5GB heap limit. I've come across that same error message - FATAL ERROR: JS Allocation failed - process out of memory - several times in my application but googling never gave me a good answer why this was happening, especially since I had a lot of spare memory on the server. Very good thing to know...

KAdot2• 11 years ago

In fact, as of this writing, the JPC depends on 103 NPM packages

It's really scary that you need so many packages to build a web app using Node.js.

danbruc3• 11 years ago

When I read 20 seconds to compile the list of the 10 most downloaded packages during the past 14 days, my first reaction was that this seems ridicules slow. How many downloads could that query have to accumulate?

A search then revealed that they have surpassed a billion downloads per month. Who on earth installs a billion packages and that every month?

NathanKP4• 11 years ago

Most companies using Node probably also have some form of CI. For example the company I work at probably generates tens of thousands of package downloads a day because we have CI running docker container builds for automated testing on every commit, and each container build downloads packages from scratch for an entirely fresh build.

Once a container is built and passes tests it is reused as many times as needed for deploying out on edge hosts so there are no more additional package downloads after that, but still the continual CI builds throughout the day as people commit code generate a lot of downloads.

aswan• 11 years ago

I think you're mixing up two different things discussed in the article. There is one remark about how many packages are used by Jut, but then the case study is about NPM using Jut to analyze traffic at the NPM registry. The billions of downloads per month is global traffic. For instance see https://twitter.com/seldo/status/631289016101441537

betenoire2• 11 years ago

considering how many "packages" are implicitly provided by a traditional LAMP stack, it's not _that_ surprising.

You want cookies? package. Sessions? package. promises? package*. make http requests? package. templating? package. hashing algorithm? package. You get the picture. If I had to replace every PHP function I use with a package, I'd end up with just as many packages.

KAdot2• 11 years ago

That's the problem. In Python, Ruby or even PHP you usually just pick a function / module from the standard library, in Node.js you need to choose one from ~10 quite similar and quite popular third-party packages.

abritinthebay• 11 years ago

Seems to me like there's very little new here to anyone paying attention to the space. They didn't raise any limits - they just fixed their own badly written code.

The 1.5 gig limit is not widely known (it's a limit of V8 and I believe, correct me if I'm wrong here, that it's improved in later versions due to some changes to GC) but if you're hitting it you're doing something massively wrong: like they were here.

Splitting data up into chunks for the event loop to process is priority one in any Node app that deals with data processing. It has been for a long time. This is true anywhere you run JavaScript. It's where libraries like [Highland](http://highlandjs.org/) excel and why Node has the concept of Buffers.

Chunking data should be a no brainer and it's frankly a little strange that Jut weren't doing this in the first place. Raises questions about what else is not being done correctly under the hood.

There's no meat here (aside from learning that NPM uses them).

Pushing the performance limits of Node.js

Related Stories