Concurrency in Ruby almost as good as Node.js

http://openhood.com/ruby/node/heroku/sinatra/mongo_mapper/unicorn/express/mongoose/cluster/2011/06/14/benchmark-ruby-versus-node-js/

by rgbrgb • 15 years ago

64 31 15 years ago

openhood.com

kevingadd• 15 years ago

Hey, look, requests/s on a synthetic benchmark.

Let's not look at things like thread safety, resource consumption, average/best case/worst case latency, quality of available libraries and their support for concurrency, or code quality/maintainability issues.

Meh.

At least he provided the source code and plenty of data. That's cool.

jshen5• 15 years ago

I await your proper and full analysis, or a link to one.

moe• 15 years ago

I don't think we are required to provide a better analysis in order to point out the problems with a flawed one.

Whatever the author is measuring here, it's most certainly not "node" vs "ruby". This should be obvious when you look at the test-apps that he's using (which involve mongodb and serialization).

alnayyir• 15 years ago

I await the death of these stupid "WELL WHY DON'T YOU MANUFACTURE YOUR OWN CAR!?" responses.

RobertLowe• 15 years ago

benchmarks like this are prolly the reason _why left us :O

maxdemarzi2• 15 years ago

This is great work but...please redo the charts to start at Zero and end at 50,000 to keep things consistent and prevent Edward Tufte from killing a kitten if he ever sees your charts.

peterquest2• 15 years ago

Ol Ed will kill kittens like nothin. I've seen him do it. He learned it from Napoleon.

peterquest2• 15 years ago

Pffff, anyone who doesn't measure things on a scale of 8747-41947 is a troglodyte.

eric-hu• 15 years ago

Yeah, the graphs are misleading because of the varying Y scale.

I do like the conclusions section, though. It presented some insightful analysis of dyno/worker efficiency

>It could be that I’m benchmarking from only 1 server but I’ve seen almost no difference between having 40 dynos or 60. You’ll see one when you receive the bill so be cautious, especially if you use an auto-scale tool.

>The same applies to node with cluster, you can do more with 15 dynos running cluster with 3 workers than with 60 dynos of node alone (for a quarter of the price!).

extension• 15 years ago

Also, you probably want to put the things you are comparing on the same chart.

cromulent• 15 years ago

I thought Tufte was OK with non-zero baselines:

"The urge to contextualize the data is a good one, but context does not come from empty vertical space reaching down to zero, a number which does not even occur in a good many data sets. Instead, for context, show more data horizontally! "

-- Edward Tufte, October 18, 2001

http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0...

maxdemarzi2• 15 years ago

A non-zero baseline is fine in some cases (line chart of the temperature of a person with a fever for example), but the issues here were multiple charts at varying Y scale starting points, and bar size comparisons of a property that we do want to go to zero (zero failures).

mrinterweb• 15 years ago

I would be interested to see how the unicorn variant, rainbows, would benchmark in this test. Rainbows seems much better suited for high concurrency applications because it utilizes fibers for concurrency as well as worker processes. http://rainbows.rubyforge.org/ Also after looking at the sinatra app's code, I believe all requests to MongoDB are going to be blocking. For the comparison to be more fair, the ruby app should be written to take advantage of Goliath, em-synchrony, event machine, or another non-blocking ruby fiber based option. Until the ruby app is non-blocking, this comparison is apples to oranges.

chaostheory• 15 years ago

Yes the nodejs concurrency model takes some time getting used to, but working with threads is much harder imo. Raw speed benchmarks don't do enough to give a full picture.

jshen5• 15 years ago

Working with threads in the context of a web request is not hard.

simpsond• 15 years ago

EventMachine uses an event loop like node, so threads are somewhat irrelevant here. Unless "workers" are threads and not actual processes, but I am pretty sure they are processes.

mhd• 15 years ago

This is comparing nodejs with Ruby's EventMachine, so where do threads enter the picture? It's been a while since I looked at EM, but if I remember correctly, it's pretty much in the same area as Python's Twisted or node.js, as opposed to something like plain ol' Java threads or Erlang.

stock_toaster• 15 years ago

I don't actually even see eventmachine in there, aside from a remark about having tried it in the opening introduction. Seems it is just comparing node with sinatra+mongomapper running under thin or unicorn.

Based on the title I assumed it was something like goliath or sinatra/async with rainbows!/thin. Looking at the git repo linked in the article though, I don't believe that is the case.

getsat• 15 years ago

Threads are inferior to events for concurrency, but they're better for throughput.

number_of_threads * thread_stack_size = sadface.jpg (when trying to use threads to handle lots of connections)

rockarage• 15 years ago

This is not a Ruby vs Node.js benchmark because Node.js is still a layer above the V8 Javascript engine. Moreover Node.js was optimize for things like concurrency and this is similar to what eventmachine is optimized for. The difference is node.js makes the event model a language construct. I think it would have been better to use eventmachine but you did not because:

> At first I tried to compare bare node.js against eventmachine_httpserver. It did quickly became obvious that this kind of micro-benchmark wasn’t going to be very helpful in deciding which one to choose.

Your first observation was the correct one. The best thing to do is just pick a language and go with the best tools available.

dreamdu5t• 15 years ago

Requests/sec is not a good benchmark, especially considering how node.js handles streaming.

For example, if he tested a 1 meg stream of chunked data being served on request, the node.js numbers would be very different.

desireco42• 15 years ago

I just can't resist, Serbs have saying 'almost is what makes girl unhappy' :).

On more serious note, ruby is definitely better when it comes to richness of available libraries which is what node is missing. I think in real life node can handle more, I heard that issues with memory leaking are solved, probably uses less memory as well. I agree with conclusion that both are more then capable or handling async problems, I would use event machine on ruby end, this whole example seems flawed a bit. Anyhow, thanks for posting your results.

ianloic• 15 years ago

But node.js doesn't have any concurrency. It's single-process, single-threaded and event driven, so by definition it's not concurrent. It's just not blocking.

Also, was someone really trying to run performance benchmarks on a shared machine running on a VM and expecting to get anything meaningful out? Really? Like really? I'm embarrassed for you.

jshen5• 15 years ago

http://stackoverflow.com/questions/1050222/concurrency-vs-pa...

tszming• 15 years ago

Even if you are right, your title should be "Concurrency in Ruby almost as good as Node.js ON Heroku"

Concurrency in Ruby almost as good as Node.js

Related Stories