Snap: A Haskell Web Framework: Benchmarks

by aliasaria • 16 years ago

44 17 16 years ago

rubyrescue• 16 years ago

interesting. i would like to see a comparison to yaws or mochiweb (perhaps w/nitrogen).

acangiano2• 16 years ago

Let me state once again that the Rails benchmarks are absolutely way off. Snap would still be faster - much faster - but you can expect several thousands req/s from that Rails app on that kind of hardware.

wwortiz• 16 years ago

It was said in another thread that these were done using development servers: http://news.ycombinator.com/item?id=1371048

Edit: Oh he responded to you the last time, but I'll leave this here so others can see

rufugee4• 16 years ago

I did my own testing, but I didn't publish them because I left Haskell completely unoptimized and didn't really have the time to try to optimize it. I also added play (http://www.playframework.org) into the mix, because I admire the framework and how closely it mimics Rails, but in Java (and Scala).

Still, I saved 'em, so here you go:

On a 2.27Ghz Xeon dual quad core with 12 GB ram, using Snap's tests from GitHub:

  Test Pong:
  | Rails Webrick | Rails Mongrel | Rails Thin  | Rails JRuby (glassfish) | Snap          | Play!         |        
  | 155.0 req/s   | 1940.5 req/s  | 619.0 req/s | 3606.0 req/s            | 11347.0 req/s | 19602.2 req/s |

  Test File:
  | Rails Webrick | Rails Mongrel | Rails Thin  | Rails JRuby (glassfish) | Snap          | Play!         |        
  | Didn't test   | 188.3 req/s   | 718.0 req/s | 6633.3 req/s            | memory leak   | 606.6 conn/s  |

The full results are here:

Pong: http://gist.github.com/414724 File: http://gist.github.com/414728

Note, Snap had a mem leak when performing the file test that caused a machine with 12 GB to swap, but that may be because I used the Ubuntu Lucid version of Haskell.

Also note, Play is damned fast. And relative to other Rails options, JRuby with glassfish is as well.

mark_l_watson• 16 years ago

Very interesting - thanks.

Scala is a hot language right now, and when Play gets really solid Scala support (does it have it already?) then Play may get to be more popular.

mightybyte• 16 years ago

You need to look at the average reply rate instead of the request rate. Requests/sec can be misleading. The replies / sec tell quite a different story. Here they are for pong:

Rails Webrick: 153.9 Rails Thin: 33.3 Rails Mongrel: 90.7 Rails JRuby (glassfish): 3,350 JRuby Mongrel: 31.3 Snap: 11,309 Play: 19,796

acangiano2• 16 years ago

If you can, try nginx + unicorn or passenger. It should be faster than JRuby on Rails with Glassfish.

warfangle• 16 years ago

Would love to see a comparison to Node.js on here, as well.

matthijs• 16 years ago

Indeed, for example the express framework appears to be optimized for high performance http://expressjs.com/

angusgr• 16 years ago

Although these seem like useful (and impressive) baseline benchmarks, won't the really important benchmarks come from measuring actual dynamic content generation, and/or caching performance for generated dynamic content?

prb• 16 years ago

When I did experiments with RoR versus Haskell via FCGI, Haskell had around a two orders of magnitude performance advantage under load and only required a single OS process.

Snap: A Haskell Web Framework: Benchmarks

Related Stories