Scaling node.js to 100k concurrent connections

http://blog.caustik.com/2012/04/08/scaling-node-js-to-100k-concurrent-connections/

by dworrad • 14 years ago

64 63 14 years ago

blog.caustik.com

ericz• 14 years ago

Before everyone gets excited about these big numbers, I would like to remind you that even higher concurrency can be achieved with even lower CPU and memory usage using Erlang. These numbers are good for Node, but don't use this as evidence that Node is magical and much better at handling large numbers of connections than other systems.

est• 14 years ago

what about other languages?

boyter• 14 years ago

I would be curious to know how something like Play! with all Async requests, or MVC with all Async would do. Probably similarly impressive, but I really have no idea.

ww520• 14 years ago

The Netty library for Java routinely can do 500K connections without any sweat. Some people have tried 1M connections with beefy machines.

jimparkins• 14 years ago

People are excited by Node doing numbers like this because there is a massive active Javascript community - with hundreds of thousands of people using Javascript all day every day at work. 0.01% of these people would ever consider learning Erlang, and even if they did they would not be able to use it at work - ever. As with everything having better features means nothing if nobody adopts. I am not saying nobody uses Erlang and I am not saying people are not adopting it - but the number are just not comparable to the Javascript community . Lastly just because you know a but of Javascript I realise that this does not mean you can architect massive real time systems. But it is like WOW even people that play casually aspire to having the best kit or playing for a top guild.

mediocregopher• 14 years ago

>As with everything having better features means nothing if nobody adopts

Well, it means my product is going to be superior since I went the better, if less well known, architecture. It's not like erlang is a little unsupported side-project of a language, it's actually older then javascript if you count the time period before it was open-sourced, and only a few years younger if you don't, and is used extensively by many industries.

Also, just because javascript the language is more well-known doesn't mean javascript the server architecture is more well-known. I would argue it isn't; when people want a highly concurrent, solid server, erlang is always mentioned.

Lastly, erlang is a pretty easy language to learn. I had the basics down in a day, I had a prototype pubsub server that could handle 50k connections in two. The syntax is a bit strange, and honestly it does get in the way sometimes, but it's not hard.

weixiyen2• 14 years ago

> they would not be able to use it at work - ever

They probably shouldn't be using node.js at work either.

Putting a client-side javascript engineer (even a decent one) on a node.js project can be really dangerous.

scarmig• 14 years ago

Interesting example from 2008:

http://www.metabrew.com/article/a-million-user-comet-applica...

I'd also add that this shouldn't be taken as something to say that Erlang is totally superior to node or that Erlang makes scaling to 1M concurrent connections a piece of cake. If you're working at that level, there's no magical out of the box solution.

ricardobeat3• 14 years ago

The magic is in javascript. How many people can write Erlang?

luriel• 14 years ago

How many people can write clean, efficient and maintainable JavaScript?

And Erlang is not the only sane option for this kinds of problems, there is also Go. And to a lesser degree you can do the same in many other languages given the right libraries and careful thinking, it takes more effort than with Erlang or Go, but almost anything beats JavaScript in both performance and code clarity (both at the 'low' code-readability level, and at the high 'project organization and design' level).

jeltz• 14 years ago

Depends on where you are. In Stockholm I would not be surprised if more people can write decent Erlang compared to the people who can write decent JavaScript.

pron• 14 years ago

... or Java.

ebiester3• 14 years ago

How fast can I get a median programmer to learn Erlang, learn the libraries, and be productive enough to be able to make these high concurrency apps?

Let's say they are a full stack programmer who knows some html, some css, some javascript, some java, and some sql.

I have a pretty good idea how fast I can bring someone up to speed on node.js -- I have to teach them some advanced JS concepts, some node.js conventions, and the APIs of my library. Async takes a little bit to wrap your head around, but it's not terrible.

Node.js seems like it is on the way to "worse is better."

rubyrescue• 14 years ago

about a week

ericmoritz3• 14 years ago

A week with the Erlang language, which will enable to write projects that would be on par or better than anything written in node.js as far as resilient and scalability.

Probably another week to get up to speed with OTP for all the promises of resilient Erlang applications.

What is often missed about Erlang is that it's not really about highly concurrent applications. That property is actually a means to accomplish its primary goal: fault-tolerant applications.

See: http://www.erlang.org/download/armstrong_thesis_2003.pdf to understand the motives of the Erlang designers.

dysinger• 14 years ago

Yawn. How about 2 million connections from a single server with Erlang & FreeBSD? http://blog.whatsapp.com/index.php/2012/01/1-million-is-so-2...

testing12341234• 14 years ago

250k concurrent connections on "... a $0.12/hr rackspace 2GB cloud server." vs 2M concurrent connections on a 24 core server with 96GB of memory.

mariuz• 14 years ago

I wonder what happens at 100k database connections , i will give a try with firebird and the nodejs driver

ricardobeat3• 14 years ago

You would be using a connection pool instead of opening one for every client.

babuskov5• 14 years ago

I have 500-600 node connections using a single DB connection and it works fine. It's MySQL using binary driver though.

bradleyland• 14 years ago

That's the thing about these types of benchmarks. They're useful for showing that node has the throughput -- at a low level -- to serve a huge number of concurrent connections, but it doesn't translate directly to huge application throughput if you're relying on things like database access over a network. In practice, each of these problems must be solved individually.

I don't mean to minimize this accomplishment. If you're assuming you need 100k database connections in order to scale, you might be solving the wrong problem. Scaling is a matter of moving data as close to the CPU as possible. This means in-memory caching is where real performance comes in. I don't care how good your language/framework is, you can't defeat the physics of slow I/O over a network.

bluesmoon• 14 years ago

i remember seeing this on HN back in April

dotborg23• 14 years ago

Looks like author is not aware of some cuncurrency problems, deadlocks etc. Backend/database might not scale to 100k concurrent connections so easily.

benologist3• 14 years ago

This is where NodeJS really starts to shine - persistant connections and background operations let you do a whole bunch of cool stuff to mitigate that.

In my case I have entire db tables and collections replicated in memory and kept in sync via redis pubsub, and the 100,000s of concurrent users I have are all sharing just a few dozen persistant redis and mongodb connections between them.

darkarmani• 14 years ago

Scaling the backend is a lot easier than dealing with concurrent front-end connections!

gaius• 14 years ago

Isn't this really scaling the underlying C runtime to 100k connections?

nicolast• 14 years ago

It struck me the author runs his apps as root (in screenshots). But then I remembered he's using node.js to handle "thousands of concurrent connections".

xentronium• 14 years ago

I think it's his testing machine, so it's shoot & forget setup.

babuskov5• 14 years ago

I use Node in production. The main thing I like about it is that looking at system usage graphs while number of users grow, only thing that is going UP is bandwidth ;)

I'd really like to see a story of someone really having 100k connected browsers. My online game currently peaks at about 1000 concurrent connections, and node process rarely lasts longer than 2 hours before it crashes. Of course, using a db like Redis to keep users sessions makes the problem almost invisible to users, as restart is instantaneous. I'm using socket.io, express, crypto module, etc.

I'd really like to see real figures for node process uptime from someone having 5000+ concurrent connections.

poundy• 14 years ago

I could never get my socket.io instance to max out, is there a good way to load test socket.io and web sockets?

giulianob2• 14 years ago

I'm using C# for my game Tribal Hero (www.tribalhero.com). It's still in early beta so I've only had 450 concurrent users . Our CPU usage and memory usage barely moved from 0 to 450 users. We're using socket selects and not even async sockets which would have even better performance. It's also backed by MySQL though we want to eventually move to Redis. Why is Node breaking at 1k connections? Doesn't seem like much at all.

babuskov5• 14 years ago

I also use MySQL as backlog, it's practically write-only as I keep the whole state in javascript objects. Only time when data is read from MySQL is at program startup. However, having SQL database enables me to run various complex SQL queries for reporting.

However, I do use Redis for one thing: user sessions. I turned persistence off as Redis seems to be rock-stable, and I really don't need sessions to persist. I was using a modified version of Node's MemoryStore, to which I added clean garbage collection, but with often restarts I mentioned earlier it has become pain for users to have to login again when in the middle of the game. Having a separate, dedicated Redis instance to handle the sessions made restarts completely seamless, as the cookie sent by user's browser remains valid between node restarts.

I was not willing to learn new db technology, but there wasn't really much to learn with Redis. You can set it up in minutes and it just works(tm). I highly recommend you try it.

benologist3• 14 years ago

I do about 300,000 - 500,000 concurrent connections on nodejs but it's all short lived web requests.

It took a while to iron out most cases that can crash, right now I have:

web.1: up for 12h

web.2: up for 12h

web.3: up for 12h

web.4: up for 12h

web.5: up for 12h

web.6: up for 4h

web.7: up for 1h

web.8: up for 12h

web.9: up for 12h

web.10: up for 12h

web.11: up for 32m

web.12: up for 12h

web.13: up for 7h

web.14: up for 7h

babuskov5• 14 years ago

Interesting. What's the system's uptime? Close to 12h or rather not? If the latter, this still means 1-2 restarts per day.

Regarding crashes, do you know of any special things to look out for? I do crash dumps and log uncaught exceptions, but sometimes node simply dies without any trace in the log files.

decad• 14 years ago

Link to his next post showing him breaking 250k - http://blog.caustik.com/2012/04/10/node-js-w250k-concurrent-...

devmach• 14 years ago

It's a shame, that he didn't mentioned about kernel tuning. Without custom settings ( like net.ipv4.tcp_mem ), i think, it's a very difficult to reach this numbers.

antihero• 14 years ago

Can uwsgi/nginx be configured similarly?

Is it common practise to have node face the web without nginx?

forgotAgain• 14 years ago

Garbage collection is disabled. How is this then relevant to any real world usage?

sootzoo• 14 years ago

He's not running with GC permanently disabled, he's only disabled the automatic GC because of the huge overhead required (claiming 1-second pauses every few seconds). He also mentions it's trivial to enable manual GC and run that via setInterval/setTimeout/what-have-you.

nivertech• 14 years ago

I did 3M/node on physical severs, 800K/node on EC2 instances.

We mostly use Erlang on server-side and node.js + CoffeScript on client-side (where they rightfully belong ;)

ericmoritz3• 14 years ago

I would really love to know what he did to tune that Rackspace VM. I had a terrible time trying to get node.js and others to get past 5,000 concurrent websocket connections on a m1.large EC2 instance or on Rackspace.

Scaling node.js to 100k concurrent connections

Related Stories