Frameworks Round 6

http://www.techempower.com/blog/2013/07/02/frameworks-round-6/

by amarsahinovic • 13 years ago

125 132 13 years ago

bhauer18• 13 years ago

Thanks for posting this, amarsahinovic.

This is the latest round of our ongoing project to benchmark the performance of fundamental services provided by modern web application frameworks. With the continued contributions of the developer community, the number of frameworks and platforms included is now 74! Round 6 also adds the often-requested plaintext test with HTTP pipelining (thanks in large part to William Glozer of Wrk fame).

The results web site also now includes a "framework overhead" tab that allows you to compare a framework versus its underlying platform. This was originally conceived by a Hacker News user named goodwink [1].

Like the round before, Round 6 gave us some technical challenges, including a period where we were unable to collect any latency data. Today, some problems persist, such as the Dart test (yes, there is server-side Dart) failing to complete for most of the tests. However, rather than continue to defer Round 6, we've decided it's better to move forward and use the round-over-round progression to iterate, fix, and improve. No problem that exists today is set in stone.

To that end, as always, I would love to hear any feedback, advice, questions, criticism, and especially contributions. And we hope you find this information helpful!

[1] https://news.ycombinator.com/item?id=5454970

simonz05• 13 years ago

Just want to say thanks for doing these tests. I think this is the first time we have such a thorough overview of the basic performance of web frameworks.

jcampbell1• 13 years ago

Luminus appears to be a clojure framework, not a php framework.

Thanks again for posting these tests.

bhauer18• 13 years ago

Oops! My mistake in classifying. Many apologies to fans of Luminus. I've just corrected this.

eranation• 13 years ago

One request: I can't help but wonder how JAX-RS / Jersey on Tomcat / Glassfish perform. JAX-RS on Java EE is a serious contender to Spring, and I really want to know how a JAX-RS service (Servlet based) is compared to Spring and to raw Servlets.

ditados2• 13 years ago

are you going to include gevent (I.e. bottle/flask + gunicorn with gevent workers, and uwsgi + gevent workers) ?

bhauer18• 13 years ago

We would certainly like to include that configuration. If you have time and are willing to contribute, we'd love to receive that as a pull request.

columbo2• 13 years ago

I'm really surprised at the performance of C#... honest no-flame question: how is possible it that Python is beating C#? I've been out of the .NET world for several years, almost a decade, but I always assumed they would be at Java level performance.

edit add: This is really fantastic work

bhauer18• 13 years ago

Some caveats to point out:

* Round 6 does not yet use SQL Server. A contributor has provided SQL Server test implementations, but we don't expect to include that until Round ~7 because it will still require some additional work. It is conceivable that the .NET MySQL driver is especially poor.

* We do not yet have a test that is expressly heavy computation with no database connectivity. I suspect that C# and .NET would perform reasonably well on such a test (isolating it from its database connectivity).

* We do not yet have Windows i7 tests. But performance would likely scale about the same as seen between Linux EC2 to Linux i7.

All that said, it's also possible that the ASP.NET implementations of our tests could benefit from additional tuning. We have had a couple subject matter experts give contributions but we'd be happy to receive more.

iaskwhy2• 13 years ago

I guess you need to change the Hardware to "Win" to see it shining.

TheAnimus• 13 years ago

Indeed, mono performance isn't great. That is the problem with C#, you need ultimately to run it on IIS for web platform deployment, sadly under anything else it is rather slow.

Also these tests are a bit unfair, as I think they give benefit in some areas to frameworks that cache by default, rather than by request.

Ultimately, what I want to know is total cost, that means hardware, licenses / support agreements, development time, developer ease of hiring.

That isn't to say these aren't interesting, but they are rather apples to oranges. As I posted in the other flamebate thread about MS stack > *, I have set a note in my calandar to help improve the example for MVC.Net (https://news.ycombinator.com/item?id=5976540). Hopefully I will be able to come then when not in an exclusive agreement, they suck I know, but it pays my mortgage!

spamizbad• 13 years ago

Even then it tends to run behind Go and Node.js in most tests. Once Round 7 includes MSSQL we'll see a more accurate picture.

matdrewin• 13 years ago

Java still beats it on Windows.

nahname2• 13 years ago

Something seems very off indeed. http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...

Avalaxy• 13 years ago

I was shocked too that ASP.NET MVC performs so poor in all these tests. C# is/can be a very fast language and both C# and ASP.NET support some awesome features that seriously increase the performance of the web app. The whole async/parallellization for example.

deelowe• 13 years ago

Thanks for continuing to do these. As far as I know, no one else is doing something like this in such a comprehensive way.

rip7472• 13 years ago

i would really like to see how CFML engines (Railo, Adobe ColdFusion) and frameworks (FW/1, CFWheels, ColdBox) stack up in this comparison.

FD: I contribute to CFWheels

bhauer18• 13 years ago

We'd love to receive these test cases as pull requests if you have the time to contribute them. :)

https://github.com/TechEmpower/FrameworkBenchmarks

rip7472• 13 years ago

checking out the documentation now. is there a google group or some place I can post questions to if I need help?

clhodapp• 13 years ago

The play-slick results seem to be missing. Was there some sort of problem?

bhauer18• 13 years ago

I had not named it uniquely. I've changed its name to "play-slick." Thanks for pointing that out!

drummingds• 13 years ago

Thanks for updating that. I noticed that the Slick version seems to outperform the regular Anorm version in most tests. Do you have any insight into why that might be? I figured it would've been the other way around.

npalli• 13 years ago

Does anyone have more info on "cpoll-cppsp"? Hearing about it for the first time, it seems to be smoking all the other frameworks.

ambiate• 13 years ago

I was actually just checking it out for myself.

http://sourceforge.net/projects/cpollcppsp/

https://github.com/xaxaxa/workspace/

Seems to be a C++ framework with dynamics in the form of C++ code using an ASP style for embedding into the resource.

camus3• 13 years ago

Nice, Some solutions are quite fast , but as soon as the database needs to perform heavy stuffs , well the perfs decrease dramatically. The really bottleneck seems to be the database in most cases. However ,some frameworks are basically quite slow.

Historiopode• 13 years ago

Wonderful, thanks for the continuous updates. I hope you will keep gaining momentum around this initiative; having a solid community-supported set of benchmarks which routinely improves would be a true asset.

By the way, I would like to offer a UX suggestion: it would be nice if you could select one or more frameworks and have them highlighted in the result listings, so as to be able to notice their relative position at a glance.

bhauer18• 13 years ago

Thanks for the feedback. The highlight is a good idea and I've created an issue for the suggestion [1]. I'll try to get that added in time for the next round.

[1] https://github.com/TechEmpower/FrameworkBenchmarks/issues/35...

VeejayRampay2• 13 years ago

For such a young language, I'm really surprised by the performance of Go there. Quite awesome.

YcombRegBroken5• 13 years ago

Round 4 of these tests is what triggered my dive into Go this past month, and it has been a revelation. If you add in the concurrency capabilities, the language simplicity, and nice balance between overly terse syntax (Python one liners...) and overly verbose typed languages (Java), the whole package is even more impressive.

The only real thing I think it is missing is a "high productivity" application framework like Rails to elevate it from "damn, I'm impressed" to "it would be ill advised to use almost anything else".

</reluctant fanboy rave>

bliker3• 13 years ago

how about http://robfig.github.io/revel/ ? I just skimmed thru their manual and it seems quite promising

Peaker18• 13 years ago

Have you tried modern ML-style languages? e.g: Haskell, OCaml, F#?

They also combine the nice conciseness of Python, with great concurrency (better than Go) and a lot more safety (less late night crashes).

skolos• 13 years ago

Check out https://github.com/jasondelponte/gokart - Rails style development with go. "The gokart gem by default combines SASS, Coffee Script, Rake, and Sprockets with Go to provide a great development environment. This environment supports test driven development (TDD) with ruby guard and jasmine. There are no external dependencies other than Go and ruby (which I expect you already have...)"

bliker3• 13 years ago

Honest question: Why are most of the frameworks that I know of in bottom half? When does speed becomes relevant?

RyanZAG2• 13 years ago

Three reasons as to the hierarchy:

1) Most popular frameworks you would encounter run on higher level dynamic languages such as PHP, Ruby or Python. These languages are a decent bit slower than performance optimized languages. The languages towards the top are all C, Java, Go, etc.

2) The frameworks towards the top are generally lower level - they do less automatic 'magic' than the more popular frameworks that handle things like validation automatically.

3) In general, it's more likely you will have heard of older frameworks than newer ones. The frameworks towards the top are generally all less than 2 years old (excluding Java servlets which have just always been decent). This means they make use of newer designs such as non-locking and asynchronous connection handling.

When does speed become relevant? It's actually always relevant as soon as your product gets traction. For example, 1 server at the top of that chart may be able to handle the same load as 10 servers towards the bottom. If you're using a framework at the bottom and paying heavily for 10 servers, you could increase your startups runway substantially by swapping. However, swapping may cost more in engineer-hours than the servers.

Basically, the relevance of speed is directly related to your margin. If your margin is high (investment banking maybe?), then speed is almost irrelevant, you can afford a whole building of servers. If you're competing heavily on mass market web apps, speed is fairly critical as you can out price competition.

bhauer18• 13 years ago

Great response, RyanZAG.

To add to this, I'd encourage you to look at the source code used to implement the tests in the higher performance frameworks. Ryan's third point is very relevant: you may find that modern frameworks leverage both high-performance platforms and much of the pragmatic thinking that was introduced a few years back in the frameworks with which you are more familiar.

Source code link: https://github.com/TechEmpower/FrameworkBenchmarks/

candybar• 13 years ago

I would go even further - performance is relevant even if your margin is high, because high baseline performance means you can trade off performance for speed of development and/or business considerations elsewhere. Scaling also doesn't come for free and it's harder, not merely more costly, to manage more servers.

camus3• 13 years ago

Basically, the relevance of speed is directly related to your margin.

True though the average PHP developper is usually cheaper than the Go or Java one. Paying people is usually more expensive than buying server instances on Amazon.

High end PHP dev is as hard as Java .. The problem is that the high end PHP frameworks are at the bottom of the list...

The frameworks towards the top are generally lower level - they do less automatic 'magic' than the more popular frameworks that handle things like validation automatically.

Not really Spring,Struts,etc ... are not really low level.

There is a paradigm shift today. There is more and more demand for realtime apps. And these cannot take advantage of http proxying or caching. So the solutions that perform badely at concurency will not be seen as relevant in the future.

bliker3• 13 years ago

Thank you for great response.

But for me all this seems quite biased towards small Frameworks/Platforms. I will hardly ever serve one JSON response form full stack framework. If I just made some C server that would do those tasks very well it would be first in this list but very far of anything of value.

Albert Einstein once wrote, “Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid.”

elsurudo• 13 years ago

I would love to see how the Goliath async server (Ruby version of Node.js is one way of looking at it - http://goliath.io) performs, as I've recently started looking into it for a part of a project.

ksec• 13 years ago

I was just about to post this. And will the next round be tested on Rails 4.0?

mhixson2• 13 years ago

The best way to ensure Goliath is tested is to submit a pull request.

We'd rather test Rails 4 than obsolete versions of Rails: https://github.com/TechEmpower/FrameworkBenchmarks/issues/35...

dom962• 13 years ago

Finally! I was disappointed when this round was postponed.

I couldn't wait to see how well my framework (Jester; https://github.com/dom96/jester) performed, not only because it was my framework but also because it is written in Nimrod. I am a bit disappointed by the results but at least there is a lot of room to improve now, I didn't have much time to properly implement concurrency for Jester so I opted for just spawning a couple of processes which is definitely not ideal.

I hope that Round 7 will go better for Jester. In the meantime I hope that you will check out the Nimrod programming language (http://nimrod-code.org) despite the results that my framework achieved.

To the team that carried out these benchmarks, thank you!

bhauer18• 13 years ago

Finally indeed! In future rounds, I hope that we can spend less time trying to resolve problems--if there are problems, we'll just post the results and move on to the next round. Better to keep iterating, and allow the experts who work with each framework resolve issues as they come up.

Thanks very much for contributing Jester to the test! I really appreciate your spirit and hope to see it improve in Round 7 as well. I'm looking forward to seeing the next pull request. :)

dom962• 13 years ago

Indeed, although people may be disappointed if their framework fails and they have to wait another month for another round of results, but then I suppose the next round will come a lot quicker if you have less issues to fix.

No problem at all. Thank you for merging my process spawning silliness. You will definitely see a lot more improvements from me :)

EDIT: A little issue I just noticed: the jester "Front-end server" is shown as "None" when in fact it's "nginx".

neya• 13 years ago

An honest confession from someone inspired by these comparisons. I've been following these benchmarks very closely right from round one and ever since I've been waiting to see my favorite framework, Ruby on Rails perform decently to some extent. I waited till the last round to see some improvements and while I DID see some marginal improvements, it wasn't as expected.

This then provoked me to do some basic math. Imagine a real world startup scenario wherein you need to your application to deliver API responses (JSON) or even ordinary HTTP responses. If you use a framework from the bottom of the list (like Rails), then you may be able to serve, say, 'X' customers on your site using your application at a time. Now, if you use something from the top of the list, you may be able to serve roughly about 20 times (20X) more number of customers on a single server. This not only means less cost, it also means you can worry about scaling and stuff much much later than you would have to, unlike on a low performant framework.

For this reason, I decided to conduct an experiment for myself. Pick up a framework from the top of the benchmarks and experiment with the learning curve and the language. Well, after carefully evaluating, I decided JAVA wasn't for me (too much of a beast). But Scala seemed to hit the sweet spot with excellent language (1 line of Scala = 10 lines of Java on an average) features along with the trust of the JVM.[1]

So, I chose Scala and started experimenting with it. I started with the play framework and was not fond of certain decisions that were made within the framework. Also, the play framework is nothing like Ruby on Rails (but it claims to be so) and was more of self-serve and also quite heavy for my taste. Hence, I went even farther up the list and chose Scalatra. Generally, framework selection is a matter of personal preference and each one should choose whatever aligns with their philosophy. For me, it was scalatra.

Frankly, I haven't been able to ship out features at the speed at which I used to, with Rails, but there was something very important that I learnt too - Don't get comfortable with full stack frameworks without understanding what goes underneath. Rails suddenly seemed like an enemy to me because it was doing everything so magically that I literally had NO IDEA until I had to implement them myself with Scalatra. For example, do you know what AES-GCM is? Do you know HOW the secure cookie store in Rails works? What algorithm it uses?? Do you know you have to change the secret pretty often??? Do you know how authentication in Rails (Devise) works??? Can you build it from scratch???? I knew none of these, until I had to implement them myself for my application on top of Scalatra. It was seriously a pain because for the first one week, I did absolutely nothing, in terms of progress with features. But later, I started loving this way of development rather than relying on 'magic' frameworks. Now 'magic' seems scary for me because I cannot actually see what's happening underneath.

So, to cut a long story short, Should you choose the fastest framework? Or should you follow the 'ship it first' policy with an average framework??

My advice - Start with a full stack framework, something like Rails or Django, but also in parallel, try to understand how every bit of your framework works and try to implement it into a micro framework based on something from the top of the benchmarks (like Go, Scala, etc.). Most importantly LEARN. Something new everyday! And slowly shift your development towards these high performant frameworks as and when you see fit.

This is just based on my experience :)

[1] 1 JVM = 10 Thins, Source: http://www.slideshare.net/pcalcado/from-a-monolithic-ruby-on...

alinajaf• 13 years ago

For example, do you know what AES-GCM is? Do you know HOW the secure cookie store in Rails works? What algorithm it uses?? Do you know you have to change the secret pretty often??? Do you know how authentication in Rails (Devise) works??? Can you build it from scratch????

If I answered yes to literally all of these questions, am I still allowed to use rails?

sanderjd• 13 years ago

Allowed and encouraged! It's good to both have standard tools that not everybody needs to reimplement differently (and often poorly) and to understand how those tools work. Everyone should take the time to get at least highly acquainted with the libraries they depend on. I suppose this was a good way for your parent to learn that lesson, but it's also a good example of why benchmarks aren't everything for frameworks; their design and community matter a lot.

nahname2• 13 years ago

I've been on the other side of that equation. Building all that stuff by hand leads to incredible waste. Bike shedding, reinventing the wheel and substandard results.

Do I like Active Record? No, not really. Do I want to fight with the entire development team to force what works best for me onto everyone else? No, not really. If we are productive with AR, I'll accept it.

gog2• 13 years ago

I had similar ambitions, and I wanted to play with the top performers, so I started looking at the Java (gemini) ecosystem.

And after an hour of surfing I remembered why I didn't follow through the last time I had this idea. The Java ecosystem is complex, I mean really complex.

Here are some of the terms I came by that look like I should grasp before trying to make something: servlet (various versions), servlet container (resin, tomcat, jetty, ...) , OSGi, jar, war, wab, jboss, glassfish, java standard/enterprise edition, where are classes being imported from, configuration files for different containers, JNDI, classpath...

Are there any resources for Java like http://mirnazim.org/writings/python-ecosystem-introduction/ for Python or http://www.phptherightway.com/ for PHP that would help someone to start navigating all those java-isms?

"Getting started" article for developing web applications with gemini would also be helpful.

krg2• 13 years ago

Hi gog,

I'm sorry you spent time trying to get started with Gemini, because it's our internal framework that isn't released yet. That would have been frustrating for you!

From our "Motivations" section: http://www.techempower.com/benchmarks/#section=motivation "Why include this Gemini framework I've never heard of?" We have included our in-house Java web framework, Gemini, in our tests. We've done so because it's of interest to us. You can consider it a stand-in for any relatively lightweight minimal-locking Java framework. While we're proud of how it performs among the well-established field, this exercise is not about Gemini. We routinely use other frameworks on client projects and we want this data to inform our recommendations for new projects.

bhauer18• 13 years ago

gog, I totally agree that the documentation for the Java ecosystem is awful. My theory is that the relevant information for today is lost under 18 years of noise.

What I'd like to see: Java should be renamed with every major version [1], in much the same way Ubuntu, OS X, and Android are named.

[1] http://tiamat.tsotech.com/rename-java

pkroll• 13 years ago

It doesn't appear that there's been any real speedup on the Ruby on Rails results for the last several rounds, so... why is that? Does nobody in the community care about speed? Is there no way to speed up Rails? Is it not clear where the bottlenecks are? Are the profiling tools not good enough? I'd sincerely like to know if someone has taken on the task and just been frustrated, or if no one has actually tried.

bhauer18• 13 years ago

We have received numerous pull requests from a spectrum of communities, but the Ruby community has been fairly quiet on that front. For example, not a whole lot has changed with the Rails test since March aside from us adding implementations of the newer test types:

https://github.com/TechEmpower/FrameworkBenchmarks/commits/m...

That said, there isn't a whole lot of code that composes the test. Here's the controller:

https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

And here is the Python script that is used to start up nginx and Unicorn for the test:

https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

And JRuby on Resin:

https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

We'd love to get more pull requests here.

melchebo• 13 years ago

I believe the '1 JVM = 10 Thins' remark you link to refers to the amount of servers they needed to run. Not the amount of code. Also Thin is a Ruby webserver, not Scala nor Java.

If the Computer Language Benchmark is any reference, 10 lines of Java ~= 8 lines of Scala : http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...

And 10 lines of Scala ~= 6 lines of Ruby : http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...

There's a pretty dramatic performance drop noted going from Scala to Ruby though.

(or 10:8:5 Java:Scala:Ruby if you wish)

continuations4• 13 years ago

cpoll-cppsp made huge performance gains from round 5 to 6. For example multiple queries went from 1,872 rps in round 5 to 7,252 in round 6. What caused that?

Also what happened to cpoll-cppsp in the plaintext test? Its performance there took a deep dive.

bhauer18• 13 years ago

I know that the contributor of those tests made several changes. We weren't able to identify why it fails dramatically on the plaintext test, but I suspect it's a combination of the two unique characteristics of that test: the use of HTTP pipelining and higher client-side concurrency. My guess is that the pipelining in particular is causing problems.

MetaCosm3• 13 years ago

I really hope you guys start doing more in-depth high-concurrency testing. This is a huge risk / problem with many frameworks and kits. It seems to be a blackhole in your tests for some reason, which is a shame because the are so well done in other regards.

Having a framework which falls down horribly at high concurrency is something people need to know about -- in how they structure and design applications and how they structure deploys. It can also help them pick frameworks that fit them better (some frameworks will do great at high concurrency, some won't).

Can't wait till you start adding high concurrency (which should NOT require any code changes from the implementation providers, you add a zeroes to the concurrency.. 100, 1000, 10000 tests).

joeblau• 13 years ago

Is there any reason why it is so slow on plain text?

bhauer18• 13 years ago

Looks like there was a problem in cpoll-cppsp: https://github.com/TechEmpower/FrameworkBenchmarks/pull/364

This will be merged in for Round 7.

antihero2• 13 years ago

How are certain languages so much faster even when pulling from the database? Surely the database would be the bottleneck here? Surely if it's something cool the driver is doing, that could be implemented in other drivers?

Also, why are some frameworks on there a bunch of times (e.g. ASP MVC)?

continuations4• 13 years ago

Surely the database would be the bottleneck here?

The idea that "it doesn't matter which language you use because DB would be your bottleneck" might have been true 10 years ago but it's definitely not true anymore.

Latest versions of MySQL and Postgresql are extremely fast. They are written in C/C++ and heavily optimized. MySQL serving data from RAM can easily do 500K+ rps on a commodity server. Even if you need to go to disks there are $200 SSD's that give you almost 100K IOPS. Put a few of them in a RAID and you're getting hundreds of thousands of IOPS for very little money.

In all likelihood application code written in Ruby/Python/PHP with frameworks piled on top of abstractions is going to be orders of magnitude slower than the DB server written in C that has undergone 15+ years of heavy optimizations.

antihero2• 13 years ago

Ok, but say we use vanilla Python WSGI with a psycopg2 (C Python PostgreSQL extension) directly. Will this come close to the performance of the C++ framework thing?

coolsunglasses• 13 years ago

I've followed yogthos' work on Luminus since the beginning.

It doesn't merit mention independently of Compojure. It's just some sane defaults for pedagogical reasons for getting started writing web apps on top of Compojure. It adds nor removes no runtime or anything else. As it currently exists, it's a Leiningen project template generator. That's about it.

Luminus is isomorphic, for the purposes of a benchmark, with Compojure.

I'm sure yogthos will be flattered when I tell him his web app templating thingy was included though.

bhauer18• 13 years ago

Sunglasses! Good to see a familiar name.

Incidentally, yogthos should be aware that Luminus was included: https://github.com/TechEmpower/FrameworkBenchmarks/pull/293

cmccabe• 13 years ago

What about Cobol on COGS?

ditados2• 13 years ago

i don't see any mention of uwsgi or gevent in these benchmarks. Deploying Python apps without properly configuring the container/webserver will skew the results considerably (probably applicable to a couple of other entries as well).

evab• 13 years ago

Can anyone explain why the "gemini" framework with an ORM outperforms raw servlets/queries in almost all the cases?

What kind of optimizations are done ?

bhauer18• 13 years ago

Evab, Gemini is our in-house framework [1] that we've included in these tests for our own learning experience. We're not obsessed with performance, but we do try to avoid problems that are easily avoided. :) For instance, where possible, we have used Java's concurrent data structures and reduced the use of locks.

A difference in these tests is that we have our own connection pool. Speed was not a main objective when we wrote it--it's so old that if my memory is correct, it predates the availability of the connection pool that now ships with the MySQL driver, which is what we've used in the Servlet tests. Nevertheless, my conjecture is that perhaps it's just a tad quicker at selecting an idle connection. Incidentally, for a while in an earlier round, the Go guys had an alternate test implementation that used a concurrent queue for connections and it performed fantastically. We've been toying with changing Gemini to do the same at some point.

For the benchmarks project, we classified ORMs as "raw" (no ORM), micro, or full. In Gemini, we have a micro ORM. The following are examples of its use in the test code [2]:

    // Get all rows from Fortunes table.
    final List<Fortune> fortunes = store.list(Fortune.class);

    // Get a random World row.
    worlds[i] = store.get(World.class, random.nextInt(DB_ROWS) + 1);

    // Run a batch of updates from a List of entity objects.
    store.putAll(Arrays.asList(worlds));

"Micro ORM" is loosely defined, but we've applied it to ORM options that offer some abstraction over plain old SQL, but aren't as comprehensive as the standard-bearers. For example, while we have a data structure that represents relationships, we do not have a higher-level query language that traverses relationships automatically nor any similar mechanism for easy relationship traversal. It turns out that across all of these frameworks, there are dozens of what we are calling Micro ORMs.

Getting back to your question, the Gemini ORM does leverage a few tricks for performance such as ReflectASM for object creation and prepared statements for queries. But there isn't a whole lot of fancy work going on.

It's not glamorous, but the most important high-performance elements we make use of in Gemini are the JVM platform and Java's concurrent data structures. As demonstrated by the likes of Undertow (the web server component of JBoss WildFly), mind-boggling performance is available on the JVM.

[1] https://groups.google.com/d/msg/framework-benchmarks/p3PbUTg...

[2] https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

mark_l_watson• 13 years ago

Disturbing results since I so frequently have used Sinatra for hosting REST APIs for customer projects. Rethink needed.

For my own projects I use Clojure + Compojure and more recently added Node.js (often with Meteor) and was pleased to see that they benchmark well. One thing I especially like about Compojure with the Hiccup HTML generating library is that I see syntax errors immediately in the editor. I suppose that Scalatra with embedded HTML would provide the same benefit.

bemurphy• 13 years ago

Maybe benchmark it yourself? I think something is amiss, it should easily 'hello, world' at 2x higher rate than Rails. Poked around the setup a little a while back but nothing jumped out.

Hunches: it's either going through webrick or not running with RACK_ENV production

bhauer18• 13 years ago

Unicorn, nginx, production environment.

https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

Frameworks Round 6

Related Stories