Python vs. Node vs. PyPy benchmarks

http://blog.kgriffs.com/2012/11/13/python-vs-node-vs-pypy-benchmarks.html

by chanux • 13 years ago

63 53 13 years ago

blog.kgriffs.com

guilloche• 13 years ago

I am surprised that node.js performs significantly better than python & pypy. Can anyone have some explanation?

afandian• 13 years ago

Money.

raphman• 13 years ago

It depends on the web framework. As the author of the benchmark shows in a later blog post [1], Python+uWSGI can be faster than Node.js

[1] http://blog.kgriffs.com/2012/12/18/uwsgi-vs-gunicorn-vs-node... (link originally posted by jdub in this discussion)

fijal2• 13 years ago

He carefully chose to use a really bad combination for pypy (or uncarefully, always assume stupidity over malice). Fast options for PyPy are cyclone.io or tornado (or eventlet or gevent on a branch I'm working on, although it's not done).

tferris5• 13 years ago

Node/V8 is a beast and will dominate web development in few years.

EDIT: Downvote me, I don't care.

zokier• 13 years ago

That'd be true if performance would trump productivity in web development. On the other hand, I don't see a lot of web-apps written in C...

tferris5• 13 years ago

Give Node s try before judging, Node gives you same productivity as other popular languages used for web development but with C-class performance

pestaa2• 13 years ago

If the web development industry moves from PHP to JavaScript as the main backend language... well, from quality perspective nothing will change. IMO.

espadrine• 13 years ago

If one company gave you a PHP job, and another a JS backend job, and that'd be all you got, would you flip a coin to pick one?

Or is this a way to say that you feel more satisfied with your work in Python than with your work in node?

lucian19002• 13 years ago

While JS is quite flawed, it's nowhere near as bad as PHP. If JS kills all usage of PHP, the world will be a better place.

malkarouri• 13 years ago

I actually don't think so, not that I have downvoted you. V8 was such a breakthrough at the time, it blew off the competition. Since then, many competitors have picked up the tricks and as shown in the bennchmark, uWSGI holds its own against Node/V8. As a result, the Node/V8 combination is not going to be so dominant in a few years in terms of performance. Another factors will come into play such as the language and the libraries available (the ecosystem). I would find it much harder to predict the dominance of the Noode/V8 ecosystem compared to that of Python or Java in a few years' time.

jessedhillon• 13 years ago

I think this is it exactly. The pace at which infrastructure, support and tools in "competing" ecosystems can be improved will outpace the rate at which people's miss can be changed about JS. There are 10-15 years of the language's, most of which were a Very Bad time for it's reputation, and only recently has the community of developers begun to land on solid development principles -- and you can hardly call those universal. I think most people fear callback spaghetti when they think of diving into someone else's JS code.

lucian19002• 13 years ago

V8 and PyPy are similarly fast runtimes, but this benchmark tests libraries against each other. Not particularly relevant, especially as at least two of them are known to have slow IO.

tferris5• 13 years ago

As good as pypy is it's no replacement for python yet.

http://stackoverflow.com/questions/12867263/why-wasnt-pypy-i...

jdub• 13 years ago

Don't miss this follow-up post, though:

http://blog.kgriffs.com/2012/12/18/uwsgi-vs-gunicorn-vs-node...

davyjones• 13 years ago

Graph 3 as a log y chart would've been nice.

rdtsc2• 13 years ago

Another set of benchmarks that shows Erlang's cowboy coming ahead on EC instances. It also compares EC2 vs real hardware and throws more contenders in the mix.

On physical hardware the race is pretty close (except node, python-gevent, but cluster node did very well).

http://eric.themoritzfamily.com/websocket-demo-results-v2.ht...

ericmoritz• 13 years ago

Yeah, because of cowboy and Erlang's design it is going to out perform a single process evented server hands-down.

  1. The evented servers are bound to one CPU, Erlang will use all CPUs 
  2. any bit of blocking code will stall the other code, even it just for a moment.

It is a little unfair to compare the two approaches.

rdtsc2• 13 years ago

I am not sure it will always under-perform with one CPU. It depends if async threads are used and if kernel poll is enabled. Erlang is designed for better low latency response at the expense of _some_ sequential slow down. I wonder if Erlang would scale slower but it will throw less errors as number of requests increases.

EDIT:

Also it would seem benchmarking Erlang and forcing it to run on a single core just like node.js is deliberately handicapping it. One of the strengths of Erlang is exactly the ability to take advantage of multiple cores.

calpaterson2• 13 years ago

I don't understand why there are no numbers on these graphs. Graphs are pretty pointless if you don't label the axes.

insaneirish• 13 years ago

Isn't labeling axes the first rule of graphing? This is bad.

rplnt• 13 years ago

It's either fixed or there's a problem on your end.

oib2• 13 years ago

Nope, I don't get any numbers/labels either. Seems like the website is incompatible with a bunch of browsers.

thomasz• 13 years ago

works with IE9 and firefox 18, not with chrome 24.

calpaterson2• 13 years ago

Great, thanks.

Egregore• 13 years ago

For Node.js it's a big difference between 1k and 64k files, what about something in between, like 32k files?

phenom• 13 years ago

Edit: pointed out in other comment

wyuenho2• 13 years ago

I'm actually quite perplexed as to why the blog author chose to use the WSGIRef which is known to be one of the slowest of the bunch according to this much more thorough benchmark:

http://nichol.as/benchmark-of-python-web-servers

Even more perplexed, why wasn't the payload and the WSGI apps sources published?

robert-zaremba• 13 years ago

Your observation is good. Author should clarify this. But in his next articles he shows benchmark against other web frameworks, which shows more justified results

fijal2• 13 years ago

That doesn't really answer why use the slowest one with PyPy (and no, uWSGI does not work for PyPy, but a lot of stuff does)

wyuenho2• 13 years ago

His articles don't justify anything. I don't know what his payloads are or how he produced them in the apps. For all I know these 2 "benchmarks" are unreproducible. The author said he wanted to try science, there's nothing scientific about these 2 articles at all.

est• 13 years ago

If you are using wsgiref don't forget the patch

    __import__('BaseHTTPServer').BaseHTTPRequestHandler.address_string = lambda x:x.client_address[0]

swah2• 13 years ago

I'd love to see something on the JVM against this.

wheaties• 13 years ago

This whole test says more to me about the performance of non-blocking MongoDB drivers than it says about the underlying performance of the frameworks/languages. The python tests lacked a non-blocking driver. Why not use the PostGRES drivers ? That would give a much more accurate and fair treatment.

And for the record, I don't care one way or the other which is "faster," just make a damned test where one isn't given such a clear leg up.

Yaggo• 13 years ago

Just FYI, in my own tests I've found Node.js performing about 50% better (req/s) than Tornado/C-python, while Tornado/pypy comes very close to Node.js' performance. Tornado is one of the fastest Python web frameworks and does not use WSGI (by default).

The point is that pypy can crunch almost as fast as Node.js, so while Node.js may set the standard, Python is not bad choice either from performance perspective.

mercuryrising• 13 years ago

Where's the source code? I'd like to see how long it takes someone who doesn't know either language to figure out what's happening in the code (a 'new' kind of response time).

ricardobeat2• 13 years ago

Red and green side-by-side on a chart... noo :(

http://jfly.iam.u-tokyo.ac.jp/color/#pallet

http://tools.medialab.sciences-po.fr/iwanthue/

swah2• 13 years ago

IMO its a little bit excessive to ask that every graph generated is color-blind safe...

Isn't there software for colorblind people to correct that? (Something that changes the palette of colors displayed...)

ricardobeat2• 13 years ago

Red & green is the stereotypical colorblind test, that's a pretty obvious combination to avoid. It's easy to choose colors that are at right angles in the color wheel, and that's not how accessibility works - you can't require every paraplegic to have a special wheelchair that can climb stairs, despite it existing...

ltbarcly3• 13 years ago

Things like this DO NOT MATTER.

Facebook was built on PHP and MySQL, both of which are awful pieces of crap.

Twitter went down like every week for a year. It was written on RoR by people who didn't really know RoR.

Meanwhile, a bunch of better engineers using better technology got their clocks cleaned while the winners eventually had so much money they could afford to hire an army of engineers to rewrite everything.

If you know Javascript really well, use Node (although I feel sorry for you). If you know Python really well, use Python (yay). If you don't know anything, well, pick something. If you are choosing a programming language because of how many QPS you can get on some trivial task, you are an idiot.

You know the graph I would like to see? A scatterplot of how good various startup founders and early team were on engineering vs finding a market fit, with the dots labeled with how rich they are now.

Python vs. Node vs. PyPy benchmarks

Related Stories