A benchmark comparison of Node.js, Tornado, and 3 Erlang servers

http://www.ostinelli.net/a-comparison-between-misultin-mochiweb-cowboy-nodejs-and-tornadoweb/

by EvanMiller • 15 years ago

95 33 15 years ago

ostinelli.net

reddittor• 15 years ago

Breaking news: Author of web framework writes benchmark to prove his web framework is the fastest.

j2d2j2d24• 15 years ago

All three Erlang frameworks beat the pants off node.js. Not just his.

j2d2j2d24• 15 years ago

I should have clarified to say that the Erlang frameworks beat both Node and Tornado thoroughly.

EvanMiller• 15 years ago

The author emphasizes the first picture (which shows Misultin delivering 3X the throughput of a Node.js server), but I think the most interesting picture is the Response Times (picture #3). Look at the purple and yellow lines. When Node.js starts overloading at 3000 reqs/sec, the average response time drops to ~300 ms (just guessing, I'm not good at reading log graphs). But when Mochiweb starts overloading at 5000 reqs/sec, the average response time is around ~50ms.

In other words, the slowest of the Erlang servers has twice the throughput of Node.js, and handles server overload much more gracefully (at least in this contrived example).

azakai• 15 years ago

I'm not sure this is surprising.

People use Python and JS-based webservers not for their performance, but because they like the languages. And those languages were designed not for speed or concurrency but for other things, like expresiveness.

Erlang was designed with performance and concurrency in mind, and if it didn't beat the pants off of Python and JS servers, I'd be shocked.

Not that that means it's the right choice for all things. Performance isn't everything, being able to write code in Python is a good enough reason to use something like Django.

cbf2• 15 years ago

Erlang was not designed with performance in mind. It's only gotten reasonably performant in it's later life.

dons• 15 years ago

Indeed. I'd really like to see the Haskell servers, warp and snap, thrown in, since the infrastructure there was actually built for raw performance.

regularfry2• 15 years ago

That's not quite fair. Erlang was designed from the start to be very, very fast at process switching and message passing. True, that design took a while to bear fruit, but my understanding (from listening to Joe Armstrong speak about it) is that being able to make the core of the language highly efficient was a goal from the start.

j2d2j2d24• 15 years ago

I agree. It's interesting to see node fans get huffy when Erlang people say, "hey, we're still #1"

But for people who already love JS, node is a dream come true. It's really fast. Not as fast as Erlang, but more than fast enough to justify building massive projects in it. Assuming you are one of those that loves, or believes in, JavaScript...

forensic3• 15 years ago

Don't you find using callbacks for everything to be a straightjacket?

cbf2• 15 years ago

I found Steve Vinoski's comment on this exercise interesting:

http://steve.vinoski.net/blog/2011/05/09/erlang-web-server-b...

ericflo• 15 years ago

The author claims that he doesn't want to test an overly-simple "hello world" benchmark, and then he does exactly that.

rfrey• 15 years ago

The author says what he means by an overly-simple "hello world" is a test of retrieval of static resources. His test is very simple but it does require a dynamic response from the server.

kingkilr• 15 years ago

Comment I left on reddit, with my benchmark numbers for the tornado server under PyPy: http://www.reddit.com/r/programming/comments/h7cs9/cowboy_mi...

megaman821• 15 years ago

I would be interested on how the Haskell frameworks like Warp or Snap stand up to Erlang.

hendzen• 15 years ago

Yesod/Warp seems to be incredibly fast according to a benchmark conducted by the Yesod devs. [1] I would also like to see a comparison with Erlang.

[1] http://www.yesodweb.com/blog/preliminary-warp-cross-language...

fingerprinter• 15 years ago

This should just fall under the 'duh' category. Quite simply, Erlang is the gold standard for scalability (horizontal and vertical) and nearly everything else is far #2.

However, choosing a tech is much more than just raw numbers. There are so many factors at play and it shouldn't be taken lightly.

That being said, we did something similar about 2 years ago and Erlang/Riak came out so far ahead of the competition for our needs and it wasn't even close.

I did go back to my notes just recently and looked again with Node in mind. Though I didn't do the same bake off we did back then, I did a thorough exercise with Node to make an evaluation and, while a cool tech, would not have made the cut. Erlang/OTP w/ Riak is not hard to program (anyone can learn), takes so much of the guess work out of scalability and is easy to debug. Node isn't quite mature enough to handle this yet and coupled with the true lack of multi-core, can't quite hang with the big boys just yet.

Erlang/OTP is clearly in a league of its own.

ozataman2• 15 years ago

Gold standard for scalability? There are certainly others. Check out numbers for Haskell:

http://snapframework.com/blog/2010/11/17/snap-0.3-benchmarks

http://www.yesodweb.com/blog/preliminary-warp-cross-language...

Would be interesting to repeat the author's experiment exactly, but in any case I would expect an order of magnitude upwards difference in performance.

Haskell gives you non-blocking IO without the hassle of callbacks, multithreaded operation and more.

justincormack• 15 years ago

I wouldnt expect an order of magnitude better on the same benchmark actually, especially as the benchmark in question is for a single core. There is clearly some scope to improve over these, but I doubt it is a factor of 10. Run the benchmarks rather than making claims with no backup.

argus• 15 years ago

Is it me or is this not a really fair comparison since the erlang frameworks would be utilizing both cpu's while node and tornado wouldn't be?

In real life a load balancer in front of tornado or node would be a far better solution than going down the erlang road in production.. support and maintenance wise + more programmers out there knowing the language would win hands down IMO...

evgen• 15 years ago

If you read the article closely you will notice that the Erlang VMs were started without SMP support enabled; everything was single-core. If you turned on Erlang SMP support then the comparison would be even more lop-sided than the example presented in the OP. In real life a hardware load balancer in front of a pool of Erlang servers would be even better :)

bergie• 15 years ago

Would be interesting to add AppServer-in-PHP: https://github.com/indeyets/appserver-in-php

It performed quite equally to Node.js on my small-scale tests (http://bergie.iki.fi/blog/php_can_perform_better_than_node-j...)

muyuu• 15 years ago

By using couch-DB you can have Erland I/O scalability and then use whatever you want for the rest of the server side logic.

If you have such a massive success that you need the rest of it also ported, then you can make this investment later on. The rest of us simply don't have the time to learn whatever is the fastest each month.

tmcneal• 15 years ago

The fact that CouchDB is written in Erlang is only part of the equation.

You're not considering that CouchDB views are written in Javascript or that CouchDB uses HTTP as its connection protocol, which has more overhead than the typical binary protocol of traditional DBMSes.

Also, avg. response time in CouchDB is directly proportional to the number of view invalidations that occur. If your data is changing a lot your views will be invalidated frequently and for those requests avg response time will drop significantly. Throw in a reduce function and re-indexing can become an order of magnitude (or more) slower.

st3fan• 15 years ago

Two comments.

These microbenchmarks don't really do it for me. It would be much more interesting to see a real test application being benchmarked. Something with users, objects, pages that have content.

It would be more fair to look at the code, complexity and performance drops when you actually build something more real.

Also, I know Erlang is awesome and super fast. But personally I would rather spend the money on a good horizontally scalable architecture and extra hardware if that means we can build apps in a more mainstream language like for example Python or Ruby.

antihero• 15 years ago

What sort of datastore would be a good pairing for this? Seeing as it doesn't matter how fast a server is if the datastore can't keep up. Redis? Riak? CouchDB?

A benchmark comparison of Node.js, Tornado, and 3 Erlang servers

Related Stories